Skip to main content
SpringerLink
Log in

Blind Image Deblurring via the Weighted Schatten p-norm Minimization Prior

  • Published:
Circuits, Systems, and Signal Processing Aims and scope Submit manuscript

Abstract

In this paper, we propose a new image blind deblurring model, based on a novel low-rank prior. As the low-rank prior, we employ the weighted Schatten p-norm minimization (WSNM), which can represent both the sparsity and self-similarity of the image structure more accurately. In addition, the L0-regularized gradient prior is introduced into our model, to extract significant edges quickly and effectively. Moreover, the WSNM prior can effectively eliminate harmful details and maintain dominant edges, to generate sharper intermediate images, which is beneficial for blur kernel estimation. To optimize the model, an efficient optimization algorithm is developed by combining the half-quadratic splitting strategy with the generalized soft-thresholding algorithm. Extensive experiments have demonstrated the validity of the WSNM prior. Our flexible low-rank prior enables the proposed algorithm to achieve excellent results in various special scenarios, such as the deblurring of text, face, saturated, and noise-containing images. In addition, our method can be extended naturally to non-uniform deblurring. Quantitative and qualitative experimental evaluations indicate that the proposed algorithm is robust and performs favorably against state-of-the-art algorithms.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23
Fig. 24
Fig. 25
Fig. 26
Fig. 27
Fig. 28
Fig. 29
Fig. 30

Similar content being viewed by others

References

  1. E.J. Candes, X. Li, Y. Ma, J. Wright, Robust principal component analysis? J. ACM 58(3), 1–37 (2011)

    Article  MathSciNet  Google Scholar 

  2. A. Chakrabarti, A neural approach to blind motion deblurring, in European Conference on Computer Vision (ECCV) (2016), pp. 221–235

  3. L. Chen, F. Fang, T. Wang, G. Zhang, Blind image deblurring with local maximum gradient prior, in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2019), pp. 1742–1750

  4. S. Cho, S. Lee, Fast motion deblurring. ACM Trans. Graph. 28(5), 1–8 (2009)

    Article  Google Scholar 

  5. S. Cho, J. Wang, S. Lee, Handling outliers in non-blind image deconvolution, in International Conference on Computer Vision (ICCV) (2011), pp. 495–502

  6. K. Dabov, A. Foi, V. Katkovnik, K. Egiazarian, Image denoising by sparse 3-D transform-domain collaborative filtering. IEEE Trans. Image Process. 16(8), 2080–2095 (2007)

    Article  MathSciNet  Google Scholar 

  7. J. Dong, J. Pan, Z. Su, Blur kernel estimation via salient edges and low rank prior for blind image deblurring. Sig. Process. Image Commun. 58, 134–145 (2017)

    Article  Google Scholar 

  8. W. Dong, G. Shi, X. Li, Image deblurring with low-rank approximation structured sparse representation, in Proceedings of The 2012 Asia Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC) (2012), pp. 1–5

  9. W. Dong, G. Shi, X. Li, Nonlocal image restoration with bilateral variance estimation: a low-rank approach. IEEE Trans. Image Process. 22(2), 700–711 (2013)

    Article  MathSciNet  Google Scholar 

  10. W. Dong, L. Zhang, G. Shi, X. Wu, Image deblurring and super-resolution by adaptive sparse domain selection and adaptive regularization. IEEE Trans. Image Process. 20(7), 1838–1857 (2011)

    Article  MathSciNet  Google Scholar 

  11. R. Fergus, B. Singh, A. Hertzmann, S.T. Roweis, W.T. Freeman, Removing camera shake from a single photograph. ACM Trans. Graph. 25(3), 787–794 (2006)

    Article  Google Scholar 

  12. S. Gu, Q. Xie, D. Meng, W. Zuo, X. Feng, L. Zhang, Weighted nuclear norm minimization and its applications to low level vision. Int. J. Comput. Vision 121(2), 183–208 (2017)

    Article  Google Scholar 

  13. S. Gu, L. Zhang, W. Zuo, X. Feng, Weighted nuclear norm minimization with application to image denoising, in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2014), pp. 2862–2869

  14. Y. Guo, H. Ma, Image blind deblurring using an adaptive patch prior. Tsinghua Sci. Technol. 24(2), 238–248 (2019)

    Article  MathSciNet  Google Scholar 

  15. M. Hirsch, C.J. Schuler, S. Harmeling, B. Schölkopf, Fast removal of non-uniform camera shake, in International Conference on Computer Vision (ICCV) (2011), pp. 463–470

  16. H. Ji, C. Liu, Z. Shen, Y. Xu, Robust video denoising using low rank matrix completion, in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2010), pp. 1791–1798

  17. K.H. Jin, J.C. Ye, Annihilating filter-based low-rank hankel matrix approach for image inpainting. IEEE Trans. Image Process. 24(11), 3498–3511 (2015)

    Article  MathSciNet  Google Scholar 

  18. N. Joshi, R. Szeliski, D.J. Kriegman, PSF estimation using sharp edge prediction, in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2008), pp. 1–8

  19. B.R. Kapuriya, D. Pradhan, R. Sharma, Detection and restoration of multi-directional motion blurred objects. SIViP 13(5), 1001–1010 (2019)

    Article  Google Scholar 

  20. R. Köhler, M. Hirsch, B. Mohler, B. Schölkopf, S. Harmeling, Recording and playback of camera shake: benchmarking blind deconvolution with a real-world database, in European Conference on Computer Vision (ECCV) (2012), pp. 27–40

  21. D. Krishnan, T. Tay, R. Fergus, Blind deconvolution using a normalized sparsity measure, in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2011), pp. 233–240

  22. O. Kupyn, V. Budzan, M. Mykhailych, D. Mishkin, J. Matas, DeblurGAN: blind motion deblurring using conditional adversarial networks, in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017), pp. 8183–8192

  23. W.-S. Lai, J.-B. Huang, Z. Hu, N. Ahuja, M.-H. Yang, A comparative study for single image blind deblurring, in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016), pp. 1701–1709

  24. A. Levin, R. Fergus, F. Durand, W.T. Freeman, Image and depth from a conventional camera with a coded aperture. ACM Trans. Graph. 26(3), 70–78 (2007)

    Article  Google Scholar 

  25. A. Levin, Y. Weiss, F. Durand, W.T. Freeman, Understanding and evaluating blind deconvolution algorithms, in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2009), pp. 1964–1971

  26. A. Levin, Y. Weiss, F. Durand, W.T. Freeman, Efficient marginal likelihood optimization in blind deconvolution, in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2011), pp. 2657–2664

  27. J. Li, W. Lu, Blind image motion deblurring with L0-regularized priors. J. Vis. Commun. Image Represent. 40, 14–23 (2016)

    Article  Google Scholar 

  28. L. Li, J. Pan, W.-S. Lai, C. Gao, N. Sang, M.-H. Yang, Blind image deblurring via deep discriminative priors. Int. J. Comput. Vision 127(8), 1025–1043 (2019)

    Article  Google Scholar 

  29. M. Li, J. Liu, Z. Xiong, X. Sun, Z. Guo, MARLow: a joint multiplanar autoregressive and low-rank approach for image completion, in European Conference on Computer Vision (ECCV), (2016), pp. 819–834

  30. L. Ma, L. Xu, T. Zeng, Low rank prior and total variation regularization for image deblurring. J. Sci. Comput. 70(3), 1336–1357 (2017)

    Article  MathSciNet  Google Scholar 

  31. J. Mairal, F. Bach, J. Ponce, G. Sapiro, A. Zisserman, Non-local sparse models for image restoration, in International Conference on Computer Vision (ICCV) (2009), pp. 2272–2279

  32. T. Michaeli, M. Irani, Blind deblurring using internal patch recurrence, in European Conference on Computer Vision (ECCV) (2014), pp. 783–798

  33. S. Nah, T. Kim, K. Lee, Deep multi-scale convolutional neural network for dynamic scene deblurring, in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017), pp. 3883–3891

  34. F. Nie, H. Huang, C. Ding, Low-rank matrix recovery via efficient Schatten p-norm minimization, in AAAI Conference on Artificial Intelligence (AAAI) (2012), pp. 655–661

  35. J. Pan, Z. Hu, Z. Su, M.-H. Yang, Deblurring text images via L0-regularized intensity and gradient prior, in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2014), pp. 2901–2908

  36. J. Pan, Z. Lin, Z. Su, M.-H. Yang, Robust kernel estimation with outliers handling for image deblurring, in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016), pp. 2800-2808

  37. J. Pan, R. Liu, Z. Su, X. Gu, Kernel estimation from salient structure for robust motion deblurring. Sig. Process. Image Commun. 28(9), 1156–1170 (2013)

    Article  Google Scholar 

  38. J. Pan, D. Sun, H. Pfister, M.-H. Yang, Blind image deblurring using dark channel prior, in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016), pp. 1628–1636

  39. D. Perrone, R. Diethelm, P. Favaro, Blind deconvolution via lower-bounded logarithmic image priors, in International Conference on Energy Minimization Methods in Computer Vision and Pattern Recognition (EMMCVPR) (2015), pp. 112–125

  40. W. Ren, X. Cao, J. Pan, X. Guo, W. Zuo, M.-H. Yang, Image deblurring via enhanced low rank prior. IEEE Trans. Image Process. 25(7), 3426–3437 (2016)

    Article  MathSciNet  Google Scholar 

  41. C.J. Schuler, M. Hirsch, S. Harmeling, B. Scholkopf, Learning to Deblur. IEEE Trans. Pattern Anal. Mach. Intell. 38(7), 1439–1451 (2016)

    Article  Google Scholar 

  42. Q. Shan, J. Jia, A. Agarwala, High-quality motion deblurring from a single image. ACM Trans. Graph. 27(3), 1–10 (2008)

    Article  Google Scholar 

  43. L. Sun, S. Cho, J. Wang, J. Hays, Edge-based blur kernel estimation using patch priors, in IEEE International Conference on Computational Photography (ICCP) (2013), pp. 1–8

  44. Y. Tang, Y. Xue, Y. Chen, L. Zhou, Blind deblurring with sparse representation via external patch priors. Digit. Signal Proc. 78, 322–331 (2018)

    Article  Google Scholar 

  45. X. Tao, H. Gao, X. Shen, J. Wang, J. Jia, Scale-recurrent network for deep image deblurring, in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2018), pp. 8174–8182

  46. O. Whyte, J. Sivic, A. Zisserman, Deblurring shaken and partially saturated images. Int. J. Comput. Vision 110(2), 185–201 (2014)

    Article  Google Scholar 

  47. O. Whyte, J. Sivic, A. Zisserman, J. Ponce, Non-uniform deblurring for shaken images. Int. J. Comput. Vis. 98(2), 168–186 (2011)

    Article  MathSciNet  Google Scholar 

  48. Y. Xie, S. Gu, Y. Liu, W. Zuo, W. Zhang, L. Zhang, Weighted Schatten p-norm minimization for image denoising and background subtraction. IEEE Trans. Image Process. 25(10), 4842–4857 (2016)

    Article  MathSciNet  Google Scholar 

  49. L. Xu, J. Jia, Two-phase kernel estimation for robust motion deblurring, in European Conference on Computer Vision (ECCV) (2010), pp. 157–170

  50. L. Xu, C. Lu, Y. Xu, J. Jia, Image smoothing via L0 gradient minimization. ACM Trans. Graph. 30(6), 1–12 (2011)

    Google Scholar 

  51. L. Xu, S. Zheng, J. Jia, Unnatural L0 sparse representation for natural image deblurring, in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2013), pp. 1107–1114

  52. Y. Yan, W. Ren, Y. Guo, R. Wang, X. Cao, Image deblurring via extreme channels prior, in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017), pp. 6978–6986

  53. Z. Zha, X. Zhang, Y. Wu, Q. Wang, X. Liu, L. Tang, X. Yuan, Non-convex weighted lp nuclear norm based ADMM framework for image restoration. Neurocomputing 311, 209–244 (2018)

    Article  Google Scholar 

  54. J. Zhang, D. Zhao, W. Gao, Group-based sparse representation for image restoration. IEEE Trans. Image Process. 23(8), 3336–3351 (2014)

    Article  MathSciNet  Google Scholar 

  55. D. Zoran, Y. Weiss, From learning models of natural image patches to whole image restoration, in International Conference on Computer Vision (ICCV) (2011), pp. 479–486

  56. W. Zuo, D. Meng, L. Zhang, X. Feng, D. Zhang, A generalized iterated shrinkage algorithm for non-convex sparse coding, in International Conference on Computer Vision (ICCV) (2013), pp. 217–224

Download references

Acknowledgements

We thank the anonymous reviewers for their valuable comments, which have helped to improve the quality of this paper. Additionally, we thank Liwen Bianji, Edanz Editing China (www.liwenbianji.cn/ac), for editing the English text of a draft of this manuscript.

Funding

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Author information

Authors and Affiliations

  1. School of Science, Nanjing University of Science and Technology, Nanjing, 210094, China

    Zhenhua Xu & Zhenhua Li

  2. Faculty of Mathematics and Physics, Huaiyin Institute of Technology, Huai’an, 223001, China

    Huasong Chen

Authors
  1. Zhenhua Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Huasong Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhenhua Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

ZX contributed to conceptualization, methodology, and writing—original draft preparation; ZX and HC performed formal analysis and investigation and provided software; ZX, HC, and ZL performed writing—review and editing; ZL helped in supervision and project administration.

Corresponding authors

Correspondence to Zhenhua Xu or Zhenhua Li.

Ethics declarations

Conflicts of interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xu, Z., Chen, H. & Li, Z. Blind Image Deblurring via the Weighted Schatten p-norm Minimization Prior. Circuits Syst Signal Process 39, 6191–6230 (2020). https://doi.org/10.1007/s00034-020-01457-z

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00034-020-01457-z

Keywords

Search

Navigation