Skip to main content
SpringerLink
Log in

XCAT - Lightweight Quantized Single Image Super-Resolution Using Heterogeneous Group Convolutions and Cross Concatenation

  • Conference paper
  • First Online:
Computer Vision – ECCV 2022 Workshops (ECCV 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13802))

Included in the following conference series:

Abstract

We propose a lightweight, single-image super-resolution mobile device network named XCAT, and introduce Heterogeneous Group Convolution Blocks with Cross Concatenations (HXBlock). The heterogeneous split of the input channels to the group convolution blocks reduces the number of operations, and cross concatenation allows for information flow between the intermediate input tensors of cascaded HXBlocks. Cross concatenations inside HXBlocks can also avoid using more expensive operations like 1 \(\times \) 1 convolutions. To further prevent expensive tensor copy operations, XCAT utilizes non-trainable convolution kernels to apply upsampling operations. Designed with integer quantization in mind, XCAT also utilizes several techniques in training, like intensity-based data augmentation. Integer quantized XCAT operates in real-time on Mali-G71 MP2 GPU with 320 ms, and on Synaptics Dolphin NPU with 30 ms (NCHW) and 8.8 ms (NHWC), suitable for real-time applications.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 149.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 199.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Ahn, N., Kang, B., Sohn, K.: Fast, accurate, and lightweight super-resolution with cascading residual network. CoRR abs/1803.08664 (2018)

    Google Scholar 

  2. Ayazoglu, M.: Extremely lightweight quantization robust real-time single-image super resolution for mobile devices. CoRR abs/2105.10288 (2021)

    Google Scholar 

  3. Ayazoglu, M.: Imdeception: grouped information distilling super-resolution network (2022)

    Google Scholar 

  4. Cogswell, M., Ahmed, F., Girshick, R., Zitnick, L., Batra, D.: Reducing overfitting in deep networks by decorrelating representations (2015)

    Google Scholar 

  5. Courbariaux, M., Bengio, Y., David, J.P.: Training deep neural networks with low precision multiplications (2015)

    Google Scholar 

  6. Dong, C., Loy, C.C., He, K., Tang, X.: Learning a deep convolutional network for image super-resolution. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8692, pp. 184–199. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10593-2_13

    Chapter  Google Scholar 

  7. Dong, C., Loy, C.C., He, K., Tang, X.: Learning a deep convolutional network for image super-resolution. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8692, pp. 184–199. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10593-2_13

    Chapter  Google Scholar 

  8. Du, Z., Liu, J., Tang, J., Wu, G.: Anchor-based plain net for mobile image super-resolution (2021)

    Google Scholar 

  9. Glasner, D., Bagon, S., Irani, M.: Super-resolution from a single image. In: 2009 IEEE 12th International Conference on Computer Vision, pp. 349–356 (2009)

    Google Scholar 

  10. He, Y., Zhang, X., Sun, J.: Channel pruning for accelerating very deep neural networks. In: 2017 IEEE International Conference on Computer Vision (ICCV), pp. 1398–1406 (2017)

    Google Scholar 

  11. Hui, Z., Gao, X., Yang, Y., Wang, X.: Lightweight image super-resolution with information multi-distillation network, pp. 2024–2032 (2019)

    Google Scholar 

  12. Hui, Z., Wang, X., Gao, X.: Fast and accurate single image super-resolution via information distillation network, pp. 723–731 (2018)

    Google Scholar 

  13. Ignatov, A., et al.: Real-time quantized image super-resolution on mobile NPUs, mobile AI 2021 challenge: Report, pp. 2525–2534 (2021)

    Google Scholar 

  14. Ignatov, A., Timofte, R., Denna, M., Younes, A., et al.: Efficient and accurate quantized image super-resolution on mobile NPUs, mobile AI & aim 2022 challenge: report. In: Proceedings of the European Conference on Computer Vision (ECCV) Workshops (2022)

    Google Scholar 

  15. Ignatov, A., et al.: AI benchmark: all about deep learning on smartphones in 2019 (2019)

    Google Scholar 

  16. Ioannou, Y., Robertson, D., Cipolla, R., Criminisi, A.: Deep roots: Improving CNN efficiency with hierarchical filter groups (2017)

    Google Scholar 

  17. Jacob, B., et al.: Quantization and training of neural networks for efficient integer-arithmetic-only inference, pp. 2704–2713 (2018)

    Google Scholar 

  18. Jain, V., Bansal, P., Kumar Singh, A., Srivastava, R.: Efficient single image super resolution using enhanced learned group convolutions (2018)

    Google Scholar 

  19. Kim, J., Lee, J.K., Lee, K.M.: Accurate image super-resolution using very deep convolutional networks. CoRR abs/1511.04587 (2015)

    Google Scholar 

  20. Kim, J., Lee, J.K., Lee, K.M.: Deeply-recursive convolutional network for image super-resolution. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1637–1645 (2016)

    Google Scholar 

  21. Krishnamoorthi, R.: Quantizing deep convolutional networks for efficient inference: a whitepaper (2018)

    Google Scholar 

  22. Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: Pereira, F., Burges, C., Bottou, L., Weinberger, K. (eds.) Advances in Neural Information Processing Systems, vol. 25. Curran Associates, Inc. (2012)

    Google Scholar 

  23. Ledig, C., et al.: Photo-realistic single image super-resolution using a generative adversarial network. CoRR abs/1609.04802 (2016)

    Google Scholar 

  24. Li, Y., Zhang, K., Timofte, R., Van Gool, L., Kong, E.A.: NTIRE 2022 challenge on efficient super-resolution: methods and results (2022)

    Google Scholar 

  25. Lim, B., Son, S., Kim, H., Nah, S., Lee, K.M.: Enhanced deep residual networks for single image super-resolution. CoRR abs/1707.02921 (2017)

    Google Scholar 

  26. Lin, M., Chen, Q., Yan, S.: Network in network (2013)

    Google Scholar 

  27. Liu, Z., et al.: Swin transformer: hierarchical vision transformer using shifted windows (2021)

    Google Scholar 

  28. Niu, B., et al.: Single image super-resolution via a holistic attention network (2020)

    Google Scholar 

  29. Schwarz Schuler, J.P., Romaní, S., Abdel-nasser, M., Rashwan, H., Puig, D.: Grouped pointwise convolutions reduce parameters in convolutional neural networks. Mendel 28, 23–31 (2022)

    Google Scholar 

  30. Shi, W., et al.: Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1874–1883 (2016)

    Google Scholar 

  31. Tai, Y., Yang, J., Liu, X.: Image super-resolution via deep recursive residual network. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2790–2798 (2017)

    Google Scholar 

  32. Vanhoucke, V., Senior, A., Mao, M.: Improving the speed of neural networks on CPUs (2011)

    Google Scholar 

  33. Wang, X., et al.: ESRGAN: enhanced super-resolution generative adversarial networks (2018)

    Google Scholar 

  34. Xie, S., Girshick, R., Dollár, P., Tu, Z., He, K.: Aggregated residual transformations for deep neural networks. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5987–5995 (2017)

    Google Scholar 

  35. Xu, S., Huang, A., Chen, L., Zhang, B.: Convolutional neural network pruning: a survey. In: 2020 39th Chinese Control Conference (CCC), pp. 7458–7463 (2020)

    Google Scholar 

  36. Yu, F., Koltun, V.: Multi-scale context aggregation by dilated convolutions (2016)

    Google Scholar 

  37. Yu, J., et al.: Wide activation for efficient and accurate image super-resolution. CoRR abs/1808.08718 (2018)

    Google Scholar 

  38. Zhang, T., Qi, G.J., Xiao, B., Wang, J.: Interleaved group convolutions. In: 2017 IEEE International Conference on Computer Vision (ICCV), pp. 4383–4392 (2017)

    Google Scholar 

  39. Zhang, X., Zhou, X., Lin, M., Sun, J.: ShuffleNet: an extremely efficient convolutional neural network for mobile devices. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6848–6856 (2018)

    Google Scholar 

  40. Zhang, Z., Wang, X., Jung, C.: DCSR: dilated convolutions for single image super-resolution. IEEE Trans. Image Process. 28(4), 1625–1635 (2019)

    Article  MathSciNet  Google Scholar 

  41. Zhao, R., Hu, Y., Dotzel, J., De Sa, C., Zhang, Z.: Building efficient deep neural networks with unitary group convolutions, pp. 11295–11304 (2019)

    Google Scholar 

  42. Zoph, B., Vasudevan, V., Shlens, J., Le, Q.V.: Learning transferable architectures for scalable image recognition. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8697–8710 (2018)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Aselsan Research, Ankara, Turkey

    Mustafa Ayazoglu & Bahri Batuhan Bilecen

Authors
  1. Mustafa Ayazoglu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bahri Batuhan Bilecen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mustafa Ayazoglu .

Editor information

Editors and Affiliations

  1. IBM Research - MIT-IBM Watson AI Lab, Massachusetts, USA

    Leonid Karlinsky

  2. Technion – Israel Institute of Technology, Haifa, Israel

    Tomer Michaeli

  3. Kyoto University, Kyoto, Japan

    Ko Nishino

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Ayazoglu, M., Bilecen, B.B. (2023). XCAT - Lightweight Quantized Single Image Super-Resolution Using Heterogeneous Group Convolutions and Cross Concatenation. In: Karlinsky, L., Michaeli, T., Nishino, K. (eds) Computer Vision – ECCV 2022 Workshops. ECCV 2022. Lecture Notes in Computer Science, vol 13802. Springer, Cham. https://doi.org/10.1007/978-3-031-25063-7_29

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-25063-7_29

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-25062-0

  • Online ISBN: 978-3-031-25063-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation