Skip to main content
SpringerLink
Log in

Faster image super-resolution by improved frequency-domain neural networks

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

Recently, most deep learning-based studies have focused on elaborately developing various types of neural networks in the spatial domain to tackle super-resolution. These methods usually have numerous parameters and require a huge amount of memory and time to train their networks. A frequency-domain neural network for super-resolution (FNNSR) has been presented. It designs a primitive neural network that can be trained using back propagation in the frequency domain. In this paper, we propose an improved FNNSR. In our method, the parameters of four quadrants in the compact weighting layer are shared. This substantially reduces the number of parameters in this layer. We use multiple convolutional layers with activations instead of a single convolution operator in FNNSR, so that the underlying features in transformed images can be learned. The Hartley transform is computed directly rather than through the Fourier transform. We design a new weighted Euclidean loss, which pays more attention to reconstructing the high-frequency part that is difficult to recover. Extensive experiments show that our method runs faster and requires fewer parameters than FNNSR. Though our method is not better than state-of-the-art methods in terms of quantitative performance, it still reveals encouraging results.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Notes

  1. https://github.com/xueshengke/IFNNSR.

References

  1. Agustsson, E., Timofte, R.: NTIRE 2017 challenge on single image super-resolution: dataset and study. In: IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 1122–1131 (2017)

  2. Bevilacqua, M., Roumy, A., Guillemot, C., Morel, M.l.A.: Low-complexity single-image super-resolution based on nonnegative neighbor embedding. In: British Machine Vision Conference, pp. 135.1–135.10 (2012)

  3. Chang, H., Yeung, D.Y., Xiong, Y.: Super-resolution through neighbor embedding. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 275–282 (2004)

  4. Ding, C., Bao, T., Karmoshi, S., Zhu, M.: Single sample per person face recognition with KPCANet and a weighted voting scheme. Signal Image Video Process. 11(7), 1213–1220 (2017)

    Article  Google Scholar 

  5. Dong, C., Loy, C.C., He, K., Tang, X.: Image super-resolution using deep convolutional networks. IEEE Trans. Pattern Anal. Mach. Intell. 38(2), 295–307 (2016)

    Article  Google Scholar 

  6. Dong, C., Loy, C.C., Tang, X.: Accelerating the super-resolution convolutional neural network. In: European Conference on Computer Vision, pp. 391–407 (2016)

    Chapter  Google Scholar 

  7. Duchon, C.E.: Lanczos filtering in one and two dimensions. J. Appl. Meteorol. 18(8), 1016–1022 (1979)

    Article  Google Scholar 

  8. Freeman, W., Jones, T., Pasztor, E.: Example-based super-resolution. IEEE Comput. Graph. Appl. 22(2), 56–65 (2002)

    Article  Google Scholar 

  9. He, C., Chen, Z., Liu, C.: Salient object detection via images frequency domain analyzing. Signal Image Video Process. 10(7), 1295–1302 (2016)

    Article  Google Scholar 

  10. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)

  11. Huang, G., Liu, Z., van der Maaten, L., Weinberger, K.Q.: Densely connected convolutional networks. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 2261–2269 (2017)

  12. Huang, J.B., Singh, A., Ahuja, N.: Single image super-resolution from transformed self-exemplars. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 5197–5206 (2015)

  13. Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., Darrell, T.: Caffe: convolutional architecture for fast feature embedding. In: ACM International Conference on Multimedia, pp. 675–678 (2014)

  14. Keys, R.: Cubic convolution interpolation for digital image processing. IEEE Trans. Acoust. Speech Signal Process. 29(6), 1153–1160 (1981)

    Article  MathSciNet  Google Scholar 

  15. Kim, J., Lee, J.K., Lee, K.M.: Accurate image super-resolution using very deep convolutional networks. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1646–1654 (2016)

  16. Kim, J., Lee, J.K., Lee, K.M.: Deeply-recursive convolutional network for image super-resolution. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1637–1645 (2016)

  17. Lai, W.S., Huang, J.B., Ahuja, N., Yang, M.H.: Fast and accurate image super-resolution with deep Laplacian pyramid networks. IEEE Trans. Pattern Anal. Mach. Intell. 1(1), 1–14 (2018)

    Google Scholar 

  18. Li, J., You, S., Robles-Kelly, A.: A frequency domain neural network for fast image super-resolution. In: International Joint Conference on Neural Networks, pp. 1–8 (2018)

  19. Li, S., Fan, R., Lei, G., Yue, G., Hou, C.: A two-channel convolutional neural network for image super-resolution. Neurocomputing 275, 267–277 (2018)

    Article  Google Scholar 

  20. Martin, D., Fowlkes, C., Tal, D., Malik, J.: A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: 8th IEEE International Conference on Computer Vision, pp. 416–423 (2001)

  21. Schulter, S., Leistner, C., Bischof, H.: Fast and accurate image upscaling with super-resolution forests. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 3791–3799 (2015)

  22. Shi, F., Cheng, J., Wang, L., Yap, P.T., Shen, D.: LRTV: MR image super-resolution with low-rank and total variation regularizations. IEEE Trans. Med. Imaging 34(12), 2459–2466 (2015)

    Article  Google Scholar 

  23. Shi, J., Li, Z., Ying, S., Wang, C., Liu, Q., et al.: MR image super-resolution via wide residual networks with fixed skip connection. IEEE J. Biomed. Health Inform. 23(3), 1129–1140 (2019)

    Article  Google Scholar 

  24. Shi, W., Caballero, J., Huszar, F., Totz, J., Aitken, A.P., Bishop, R., Rueckert, D., Wang, Z.: Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1874–1883. IEEE (2016)

  25. Sun, J., Xu, Z., Shum, H.Y.: Gradient profile prior and its applications in image super-resolution and enhancement. IEEE Trans. Image Process. 20(6), 1529–1542 (2011)

    Article  MathSciNet  Google Scholar 

  26. Tai, Y., Yang, J., Liu, X., Xu, C.: MemNet: a persistent memory network for image restoration. In: IEEE International Conference on Computer Vision, pp. 4549–4557 (2017)

  27. Tang, Z., Luo, L., Peng, H., Li, S.: A joint residual network with paired ReLUs activation for image super-resolution. Neurocomputing 273, 37–46 (2018)

    Article  Google Scholar 

  28. Tao, H., Lu, X.: Contour-based smoky vehicle detection from surveillance video for alarm systems. Signal Image Video Process. 13, 1–9 (2018)

    Google Scholar 

  29. Timofte, R., de Smet, V., van Gool, L.: A+: adjusted anchored neighborhood regression for fast super-resolution. In: Asian Conference on Computer Vision, pp. 111–126 (2015)

    Chapter  Google Scholar 

  30. Wang, Z., Bovik, A., Sheikh, H., Simoncelli, E.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)

    Article  Google Scholar 

  31. Xue, S., Jin, X.: Robust classwise and projective low-rank representation for image classification. Signal Image Video Process. 12(1), 107–115 (2018)

    Article  Google Scholar 

  32. Xue, S., Qiu, W., Liu, F., Jin, X.: Low-rank tensor completion by truncated nuclear norm regularization. In: 24th International Conference on Pattern Recognition, pp. 2600–2605 (2018)

  33. Yan, Q., Xu, Y., Yang, X., Nguyen, T.Q.: Single image super-resolution based on gradient profile sharpness. IEEE Trans. Image Process. 24(10), 3187–3202 (2015)

    Article  MathSciNet  Google Scholar 

  34. Yang, C., Ye, M., Tang, S., Xiang, T., Liu, Z.: Semi-supervised low-rank representation for image classification. Signal Image Video Process. 11(1), 73–80 (2017)

    Article  Google Scholar 

  35. Yang, W., Tian, Y., Zhou, F., Liao, Q., Chen, H., Zheng, C.: Consistent coding scheme for single-image super-resolution via independent dictionaries. IEEE Trans. Multimed. 18(3), 313–325 (2016)

    Article  Google Scholar 

  36. Zeyde, R., Elad, M., Protter, M.: On single image scale-up using sparse-representations. In: International Conference on Curves and Surfaces, pp. 711–730 (2012)

    Chapter  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. College of Information Science and Electronic Engineering, Zhejiang University, Hangzhou, 310027, People’s Republic of China

    Shengke Xue, Wenyuan Qiu, Fan Liu & Xinyu Jin

Authors
  1. Shengke Xue
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wenyuan Qiu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Fan Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xinyu Jin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shengke Xue.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xue, S., Qiu, W., Liu, F. et al. Faster image super-resolution by improved frequency-domain neural networks. SIViP 14, 257–265 (2020). https://doi.org/10.1007/s11760-019-01548-8

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-019-01548-8

Keywords

Search

Navigation