Region-based weighted-norm with adaptive regularization for resolution enhancement

https://doi.org/10.1016/j.dsp.2011.02.005Get rights and content

Abstract

As digital video cameras spread rapidly, critical locations such as airports, train stations, military compounds and airbases and public “hot-spots” are placed in the spotlight of video surveillance. Video surveillance provides visual information, in order to maintain security at the monitored areas. Indeed, super-resolution (SR) technique can be useful for extracting additional information from the captured video sequences. Generally, video surveillance cameras capture moving objects while cameras are moving in some cases, which means that many of the existing SR algorithms, that cannot cope with moving objects, are not applicable in this case. In this paper, we propose a SR algorithm that takes into account inaccurate registration at the moving regions and therefore copes with moving objects. We propose to adaptively weight each region according to its reliability where regions that have local motion and/or occlusion have different registration error level. Also, the regularization parameter is simultaneously estimated for each region. The regions are generated by segmenting the reference frame using watershed segmentation. Our approach is tested on simulated and real data coming from videos with different difficulties taken by a hand-held camera. The experimental results show the effectiveness of the proposed algorithm compared to four state-of-the-art SR algorithms.

Section snippets

Osama A. Omer received his B.Eng. and M.Eng. degrees in electrical engineering from South Valley University, Aswan, Egypt, in 2000 and 2004, respectively. He received his Ph.D. degree from Tokyo University of Agriculture and Technology in 2009. He is now an Assistant Professor at South Valley University. He spent the summer internship 2008 in Nokia/Tokyo research center. His interests include image/video super-resolution, image/video compression, and applications of neural networks in signal

References (20)

  • J.R. Bergen et al.

    Hierarchical model-based motion estimation

  • B. Lucas, T. Kanade, An iterative image registration technique with an application to stereo vision, in: Proc. of the...
  • M. Elad et al.

    A fast super-resolution reconstruction algorithm for pure transnational motion and common space invariant blur

    IEEE Trans. Image Process.

    (August 2001)
  • S. Farsiu et al.

    Fast and robust multi-frame super-resolution

    IEEE Trans. Image Process.

    (October 2004)
  • E.S. Lee et al.

    Regularized adaptive high-resolution image reconstruction considering inaccurate subpixel registration

    IEEE Trans. Image Process.

    (July 2003)
  • M.K. Park et al.

    Regularized super-resolution image reconstruction considering inaccurate motion information

    SPIE Optical Eng.

    (November 2007)
  • H. He et al.

    An image super-resolution algorithm for different error levels per frame

    IEEE Trans. Image Process.

    (March 2006)
  • M. Trimeche et al.

    Adaptive outlier rejection in image super-resolution

    EURASIP J. Appl. Signal Process.

    (2006)
  • Z.A. Ivanovski, L. Panovski, L.J. Karam, Robust super-resolution based on pixel-level selectivity, in: Proc. of...
  • O.A. Omer, T. Tanaka, Multiframe image and video super-resolution algorithm with inaccurate motion registration errors...
There are more references available in the full text version of this article.

Cited by (15)

  • Multiscale self-similarity and sparse representation based single image super-resolution

    2017, Neurocomputing
    Citation Excerpt :

    The prominent advantage of TV is that it tends to have a better edge preservation than some other regularization model, such as Tikhonov regularization [17], but it is quite liable to produce some unpleasing pseudo-edges in the smooth regions, especially at high noise levels. Thus, a variety of derivatives of TV are proposed to improve its performance from some particular aspects, e.g., bilateral TV [18], adaptive TV [19], locally adaptive BTV [20–23], etc. However, it has been pointed out that the performance of this kind of methods all degrades dramatically under three circumstances where (a) the amount of LR inputs is inadequate; (b) the estimate of motion is imprecise; or (c) the scale factor increases [24,25].

  • A new denoising model for multi-frame super-resolution image reconstruction

    2017, Signal Processing
    Citation Excerpt :

    After the first work proposed in [19], where the authors considered a frequency domain approach, several approaches have been proposed and studied to improve the multi-frame SR problem [20–24]. Earlier works on SR algorithms are based on regularization method due to its ill-posed nature which mainly contains the likelihood and prior function [25,26]. The likelihood function measures the difference between the LR images and the obtained HR one, while the image prior function, impose some prior knowledge on the desired HR image.

  • Single-image super-resolution reconstruction based on global non-zero gradient penalty and non-local Laplacian sparse coding

    2014, Digital Signal Processing: A Review Journal
    Citation Excerpt :

    Therefore, the signal processing methods are selected to reconstruct potential details and features hidden in the low resolution (LR) image. Generally, the existing methods can be classified into three categories: interpolation-based methods [2–5], regularization-based methods [6–12] and example-based methods [13–32]. However, the interpolation-based methods are usually prone to yield overly smooth images with ringing and jagged artifacts when a larger magnification ratio (such as a factor of more than double) is performed.

View all citing articles on Scopus

Osama A. Omer received his B.Eng. and M.Eng. degrees in electrical engineering from South Valley University, Aswan, Egypt, in 2000 and 2004, respectively. He received his Ph.D. degree from Tokyo University of Agriculture and Technology in 2009. He is now an Assistant Professor at South Valley University. He spent the summer internship 2008 in Nokia/Tokyo research center. His interests include image/video super-resolution, image/video compression, and applications of neural networks in signal processing.

Toshihisa Tanaka received the B.E., the M.E., and the Ph.D. degrees from the Tokyo Institute of Technology in 1997, 2000, and 2002, respectively. From 2000 to 2002, he was a JSPS Research Fellow. From October 2002 to March 2004, he was a Research Scientist at RIKEN Brain Science Institute. In April 2004, he joined Department of Electrical and Electronic Engineering, Tokyo University of Agriculture and Technology, where he is currently an Associate Professor. His research interests include image and signal processing, multirate systems, blind signal separation, brain signal processing, and adaptive signal processing. In 2005, he was a Royal Society Visiting Fellow at the Communications and Signal Processing Group, Imperial College London, U.K. He is a co-editor of Signal Processing Techniques for Knowledge Extraction and Information Fusion (with Mandic, Splinger), 2008. He has been a member of the Technical Committee on Blind Signal Processing, IEEE Circuits and Systems Society. He is a chair of the Technical Committee on Biomedical Signal Processing, APSIPA. He is a senior member of IEEE, and a member of IEICE and APSIPA.

View full text