Skip to main content
SpringerLink
Log in

Single image super-resolution under multi-frame method

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

The multi-frame image super-resolution method utilizes a series of low-resolution images from the same scene to reconstruct the corresponding high-quality super-resolution image. However, the insufficient input low-resolution images make it incapable of reconstructing the high-resolution image from a single low-resolution image. In this work, we extend the framework of multi-frame image super-resolution to handle the single low-resolution image via rolling guidance filtering. And an improved version of diffusion-driven regularizer-based multi-frame image super-resolution algorithm is proposed and applied on passive millimeter-wave (PMMW) image super-resolution. Specifically, the joint filtering is first exploited to suppress the noise of single low-resolution noisy image. The rolling guidance method is exploited to generate the structurally multi-scale low-resolution images forming the basis of multi-frame image super-resolution. The generated image sequences are then fed to the nonlinear diffusion regularizer-based super-resolution algorithm. The two-directional total variation de-convolution is finally employed to remove the blur, producing a sharp and clear high-resolution image. Experiments demonstrate the effectiveness of the proposed method and show its superiority for the natural and PMMW images.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Notes

  1. Some codes are available at http://lcav.epfl.ch/software/superresolution.

References

  1. Park, S.C., Park, M.K., Kang, M.G.: Super-resolution image reconstruction: a technical overview. IEEE Signal Process. Mag. 20(3), 21–36 (2003)

    Article  Google Scholar 

  2. Tschumperle, D., Deriche, R.: Vector-valued image regularization with pdes: a common framework for different applications. IEEE Trans. Pattern Anal. Mach. Intell. 27(4), 506–517 (2005)

    Article  Google Scholar 

  3. Baker, S., Kanade, T.: Limits on super-resolution and how to break them. IEEE Trans. Pattern Anal. Mach. Intell. 24(9), 1167–1183 (2002)

    Article  Google Scholar 

  4. Tsai, R., Huang, T.S.: Multiframe image restoration and registration. Adv. Comput. Vis. Image Process. 1(2), 317–339 (1984)

    Google Scholar 

  5. Glasner, D., Bagon, S., Irani, M.: Super-resolution from a single image. In: 2009 IEEE 12th International Conference on Computer Vision, 2009, pp. 349–356

  6. Sundar, K.J.A., Vaithiyanathan, V.: Multi-frame super-resolution using adaptive normalized convolution. Signal, Image Video Process. 11(2), 357–362 (2017)

    Article  Google Scholar 

  7. Tom, B.C., Katsaggelos, A.K.: Reconstruction of a high-resolution image from multiple-degraded misregistered low-resolution images. In: Visual Communications and Image Processing’94, 1994, pp. 971–981

  8. Hardie, R.C., Barnard, K.J., Armstrong, E.E.: Joint map registration and high-resolution image estimation using a sequence of undersampled images. IEEE Trans. Image Process. 6(12), 1621–1633 (1997)

    Article  Google Scholar 

  9. Liu, X., Chen, L., Wang, W., Zhao, J.: Robust multi-frame super-resolution based on spatially weighted half-quadratic estimation and adaptive BTV regularization. IEEE Trans. Image Process. (2018)

  10. Farsiu, S., Elad, M., Milanfar, P.: Multiframe demosaicing and super-resolution of color images. IEEE Trans. Image Process. 15(1), 141–159 (2006)

    Article  Google Scholar 

  11. Aguena, M.L., Mascarenhas, N.D.: Multispectral image data fusion using pocs and super-resolution. Comput. Vis. Image Underst. 102(2), 178–187 (2006)

    Article  Google Scholar 

  12. Yang, X., Zhang, Y., Zhou, D., Yang, R.: An improved iterative back projection algorithm based on ringing artifacts suppression. Neurocomputing 162, 171–179 (2015)

    Article  Google Scholar 

  13. Fang, J., Li, J., Shen, Y., Li, H., Li, S.: Super-resolution compressed sensing: an iterative reweighted algorithm for joint parameter learning and sparse signal recovery. IEEE Signal Process. Lett. 21(6), 761–765 (2014)

    Article  Google Scholar 

  14. Yang, J., Wright, J., Huang, T.S., Ma, Y.: Image super-resolution via sparse representation. IEEE Trans. Image Process. 19(11), 2861–2873 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  15. An, L., Bhanu, B.: Image super-resolution by extreme learning machine. In: 19th IEEE International Conference on Image processing (ICIP), 2012, pp. 2209–2212

  16. Dong, C., Loy, C.C., He, K., Tang, X.: Image super-resolution using deep convolutional networks. IEEE Trans. Pattern Anal. Mach. Intell. 38(2), 295–307 (2016)

    Article  Google Scholar 

  17. Huang, Y., Wang, W., Wang, L.: Video super-resolution via bidirectional recurrent convolutional networks. IEEE Trans. Pattern Anal. Mach. Intell. 40(4), 1015–1028 (2018)

    Article  Google Scholar 

  18. Polatkan, G., Zhou, M., Carin, L., Blei, D., Daubechies, I.: A bayesian nonparametric approach to image super-resolution. IEEE Trans. Pattern Anal. Mach. Intell. 37(2), 346–358 (2015)

    Article  Google Scholar 

  19. Li, L., Li, C., Yang, J.: Super-Resolution Restoration and Image Reconstruction for Passive Millimeter Wave Imaging. INTECH Open Access Publisher (2012)

  20. Zhu, S., Li, Y., Chen, J., Li, Y.: Passive millimeter wave image denoising based on adaptive manifolds. Prog. Electromagn. Res. B 57, 63–73 (2014)

    Article  Google Scholar 

  21. Zhang, Q., Shen, X., Xu, L., Jia, J.: Rolling guidance filter. In: European Conference on Computer Vision. Springer, New York, pp. 815–830 (2014)

  22. Tomasi, C., Manduchi, R.: Bilateral filtering for gray and color images. In: Sixth International Conference on Computer Vision, 1998, pp. 839–846

  23. Gastal, E.S., Oliveira, M.M.: Domain transform for edge-aware image and video processing. In: ACM Transactions on Graphics (TOG), vol. 30, no. 4.ACM, 2011, p. 69

  24. He, K., Sun, J., Tang, X.: Guided image filtering. In: European conference on computer vision. Springer, pp. 1–14 (2010)

  25. Singh, P.P., Garg, R.: Fixed point ica based approach for maximizing the non-gaussianity in remote sensing image classification. J. Indian Soc. Remote Sens. 43(4), 851–858 (2015)

    Article  Google Scholar 

  26. Singh, P.P., Garg, R.D.: A hybrid approach for information extraction from high resolution satellite imagery. Int. J. Image Graph. 13(02), 1340007 (2013)

    Article  Google Scholar 

  27. Ogada, E.A., Guo, Z., Wu, B.: An alternative variational framework for image denoising. In: Abstract and Applied Analysis, vol. 2014. Hindawi Publishing Corporation (2014)

  28. Maiseli, B.J., Elisha, O.A., Gao, H.: A multi-frame super-resolution method based on the variable-exponent nonlinear diffusion regularizer. EURASIP J. Image Video Process. 2015(1), 22 (2015)

    Article  Google Scholar 

  29. Pipa, D.R., Chan, S.H., Nguyen, T.Q.: Directional decomposition based total variation image restoration. In: Proceedings of the 20th European Signal Processing Conference (EUSIPCO), 2012, pp. 1558–1562

  30. Patti, A.J., Sezan, M.I., Tekalp, A.M.: Superresolution video reconstruction with arbitrary sampling lattices and nonzero aperture time. IEEE Trans. Image Process. 6(8), 1064–1076 (1997)

    Article  Google Scholar 

  31. Irani, M., Peleg, S.: Improving resolution by image registration. CVGIP. Graph. Models Image Process. 53(3), 231–239 (1991)

    Article  Google Scholar 

  32. Pham, T.Q., Van Vliet, L.J., Schutte, K.: Robust fusion of irregularly sampled data using adaptive normalized convolution. EURASIP J. Adv. Signal Process. 2006(1), 083268 (2006)

    Article  Google Scholar 

  33. Rudin, L.I., Osher, S.: Total variation based image restoration with free local constraints. In: Image Proceedings. ICIP-94., IEEE International Conference, vol. 1, 1994, pp. 31–35

  34. Köhler, T., Huang, X., Schebesch, F., Aichert, A., Maier, A., Hornegger, J.: Robust multiframe super-resolution employing iteratively re-weighted minimization. IEEE Trans. Comput. Imaging 2(1), 42–58 (2016)

    Article  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Electronic Engineering and Optoelectronic Technology, Nanjing University of Science and Technology, Nanjing, China

    Shujin Zhu & Yuehua Li

Authors
  1. Shujin Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yuehua Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yuehua Li.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhu, S., Li, Y. Single image super-resolution under multi-frame method. SIViP 13, 331–339 (2019). https://doi.org/10.1007/s11760-018-1361-2

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-018-1361-2

Keywords

Search

Navigation