Skip to main content
SpringerLink
Log in

Optimal Selection of the Regularization Function in a Weighted Total Variation Model. Part II: Algorithm, Its Analysis and Numerical Tests

  • Published:
Journal of Mathematical Imaging and Vision Aims and scope Submit manuscript

Abstract

Based on the weighted total variation model and its analysis pursued in Hintermüller and Rautenberg 2016, in this paper a continuous, i.e., infinite dimensional, projected gradient algorithm and its convergence analysis are presented. The method computes a stationary point of a regularized bilevel optimization problem for simultaneously recovering the image as well as determining a spatially distributed regularization weight. Further, its numerical realization is discussed and results obtained for image denoising and deblurring as well as Fourier and wavelet inpainting are reported on.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Adams, R.A., Fournier, J.J.F.: Sobolev Spaces. Volume 140 of Pure and Applied Mathematics. Elsevier/Academic Press, Amsterdam (2003)

  2. Almansa, A., Ballester, C., Caselles, V., Haro, G.: A TV based restoration model with local constraints. J. Sci. Comput. 34(3), 209–236 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  3. Athavale, P., Jerrard, R., Novaga, M., Orlandi, G.: Weighted TV minimization and applications to vortex density models. Technical report, University of Pisa, Department of Mathematics, (2015)

  4. Attouch, H., Buttazzo, G., Michaille, G.: Variational analysis in Sobolev and BV spaces. MPS-SIAM, (2006)

  5. Barbu, V.: Optimal control of variational inequalities. Res, vol. 100. Notes Math. Pitman, London, United Kingdom (1984)

  6. Bertalmio, M., Caselles, V., Rougé, B., Solé, A.: TV based image restoration with local constraints. J. Sci. Comput. 19, 95–122 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  7. Bertsekas, D.P.: On the Goldstein-Levitin-Polyak gradient projection method. IEEE Trans. Autom. Control AC–21(2), 174–184 (1976)

    Article  MathSciNet  MATH  Google Scholar 

  8. Bertsekas, D.P.: Projected Newton methods for optimization problems with simple constraints. SIAM J. Control Optim. 20(2), 221–246 (1982)

    Article  MathSciNet  MATH  Google Scholar 

  9. Bertsekas, D.P. Gafni, E.M.: Convergence of a gradient projection method. Report P-121, Laboratory for Information and Decision Systems Report, Laboratory for Information and Decision Systems, Massachusetts Institute of Technology, Cambridge, MA, (1982)

  10. Brézis, H.: Problèmes Unilatéraux. PhD thesis, Sc. math. Paris VI. 1971., (1972)

  11. Bui-Thanh, T., Willcox, K., Ghattas, O.: Model reduction for large-scale systems with high-dimensional parametric input space. SIAM J. Sci. Comput. 30(6), 3270–3288 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  12. Cao, V.C., De los Reyes, J. C., Schoenlieb, C.B.: Learning optimal spatially-dependent regularization parameters in total variation image restoration. ArXiv e-prints, Mar. (2016)

  13. Chan, R.H., Yang, J., Yuan, X.: Alternating direction method for image inpainting in wavelet domain. SIAM J. Imaging Sci. 4, 807–826 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  14. Chan, T.F., Shen, J., Zhou, H.-M.: Total variation wavelet inpainting. J. Math. Imaging Vis. 25, 107–125 (2006)

    Article  MathSciNet  Google Scholar 

  15. Chen, K., Dong, Y., Hintermüller, M.: A nonlinear multigrid solver with line Gauss-Seidel-semismooth-Newton smoother for the Fenchel pre-dual in total variation based image restoration. Inverse Probl. Imaging 5(2), 323–339 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  16. Chipot, M.: Variational Inequalities and Flow in Porous Media. Springer, New York (1984)

    Book  MATH  Google Scholar 

  17. Combettes, P.L., Pesquet, J.-C.: Proximal splitting methods in signal processing. In: Fixed-point algorithms for inverse problems in science and engineering, volume 49 of Springer Optim. Appl., pp.185–212. Springer, New York, (2011)

  18. De los Reyes, J.C., Schönlieb, C.-B., Valkonen, T.: Bilevel parameter learning for higher-order total variation regularisation models. Journal of Mathematical Imaging and Vision, pages 1–25, (2016)

  19. De Los Reyes, J.C., Schönlieb, C.-B., Valkonen, T.: The structure of optimal parameters for image restoration problems. J. Math. Anal. Appl. 434(1), 464–500 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  20. Deledalle, C.-A., Vaiter, S., Fadili, J., Peyré, G.: Stein Unbiased GrAdient estimator of the Risk (SUGAR) for multiple parameter selection. SIAM J. Imaging Sci. 7(4), 2448–2487 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  21. Dong, Y., Hintermüller, M., Rincon-Camacho, M.: Automated regularization parameter selection in multi-scale total variation models for image restoration. J. Math. Imaging Vis. 40(1), 82–104 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  22. Dong, Y., Hintermüller, M., Rincon-Camacho, M.: A multi-scale vectorial l\(^{\tau }\)-TV framework for color image restoration. Int. J. Comput. Vis. 92(3), 296–307 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  23. Dong, Y., Hintermüller, M., Rincon-Camacho, M.M.: Automated regularization parameter selection in multi-scale total variation models for image restoration. J. Math. Imaging Vis. 40(1), 82–104 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  24. Frick, K., Marnitz, P., Munk, A.: Statistical multiresolution Dantzig estimation in imaging: fundamental concepts and algorithmic framework. Electron. J. Stat. 6, 231–268 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  25. Grisvard, P.: Elliptic Problems in Nonsmooth Domains. Volume 24 of Monographs and Studies in Mathematics. Pitman. Advanced Publishing Program, Boston, MA (1985)

  26. Gumbel, E.: Les valeurs extrêmes des distributions statistiques. Ann. Inst. H. Poincaré 5(2), 115–158 (1935)

    MathSciNet  MATH  Google Scholar 

  27. Gumbel, E.J.: Statistics of extremes. Dover Publications, Inc., Mineola, NY, 2004. Reprint of the 1958 original [Columbia University Press, New York; MR0096342]

  28. Haber, E., Tenorio, L.: Learning regularization functionals—a supervised training approach. Inverse Probl. 19(3), 611–626 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  29. Hintermüller, M., Kopacka, I.: Mathematical programs with complementarity constraints in function space: \(C\)- and strong stationarity and a path-following algorithm. SIAM J. Optim. 20(2), 868–902 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  30. Hintermüller, M., Kunisch, K.: Path-following methods for a class of constrained minimization problems in function space. SIAM J. Optim. 17(1), 159–187 (2006). (electronic)

    Article  MathSciNet  MATH  Google Scholar 

  31. Hintermüller, M., Rautenberg, C.N.: Optimal selection of the regularization function in a generalized total variation model. Part I: Modelling and theory. WIAS Preprint No. 2235, (2016)

  32. Hintermüller, M., Rautenberg, C.N.: On the density of classes of closed convex sets with pointwise constraints in Sobolev spaces. J. Math. Anal. Appl. 426(1), 585–593 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  33. Hintermüller, M., Rincon-Camacho, M.: Expected absolute value estimators for a spatially adapted regularization parameter choice rule in L1-TV-based image restoration. Inverse Probl. 26(8), 085005 (2010)

    Article  MATH  Google Scholar 

  34. Hintermüller, M., Surowiec, T.M., Mordukhovich, B.S.: Several approaches for the derivation of stationarity conditions for elliptic MPECs with upper-level control constraints. Math. Program. 146(1–2), 555–582 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  35. Hintermüller, M., Wu, T.: Bilevel optimization for calibrating point spread functions in blind deconvolution. Inverse Probl. Imaging 9(4), 1139–1169 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  36. Hinze, M., Pinnau, R., Ulbrich, M., Ulbrich, S.: Optimization with PDE Constraints. Mathematical Modelling: Theory and Applications, volume 23. Springer, New York (2009)

    MATH  Google Scholar 

  37. Hotz, T., Marnitz, P., Stichtenroth, R., Davies, L., Kabluchko, Z., Munk, A.: Locally adaptive image denoising by a statistical multiresolution criterion. Comput. Stat. Data Anal. 56(3), 543–558 (2012)

    MathSciNet  MATH  Google Scholar 

  38. Jalalzai, K.: Regularization of inverse problems in image processing. Ph.D. thesis, Ecole Polytechnique (2012)

  39. Kinderlehrer, D., Stampacchia, G.: An introduction to Variational Inequalities and Their Applications. SIAM, Philadelphia (2000)

    Book  MATH  Google Scholar 

  40. Kunisch, K., Pock, T.: A bilevel optimization approach for parameter learning in variational models. SIAM J. Imaging Sci. 6, 938–983 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  41. Luo, T., Pang, J.-S., Ralph, D.: Mathematical Programs with Equilibrum Constraints. Cambridge University Press, Cambridge (1996)

    Book  Google Scholar 

  42. Nittka, R.: Elliptic and parabolic problems with Robin boundary conditions on Lipschitz domains. Ph.D. thesis, Universität Ulm (2010)

  43. Nittka, R.: Quasilinear elliptic and parabolic Robin problems on Lipschitz domains. NoDEA Nonlinear Differ. Equ. Appl. 20(3), 1125–1155 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  44. Outrata, J., Kocvara, M., Zowe, J.: Nonsmooth Approach to Optimization Problems with Equilibrium Constraints. Nonconvex Optimization and its Applications, vol. 28. Kluwer Academic, Dordrecht (1998)

    Book  MATH  Google Scholar 

  45. Pesquet, J.-C., Benazza-Benyahia, A., Chaux, C.: A SURE approach for digital signal/image deconvolution problems. IEEE Trans. Signal Process. 57(12), 4616–4632 (2009)

    Article  MathSciNet  Google Scholar 

  46. Rodrigues, J.F.: Obstacle Problems in Mathematical Physics. North-Holland, Amsterdam (1987)

    MATH  Google Scholar 

  47. Schönlieb, C., De Los Reyes, J.C.: Image denoising: learning noise distribution via PDE-constrained optimisation. Inverse Probl. Imaging 7(4), 1183–1214 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  48. Serrin, J.: Local behavior of solutions of quasi-linear equations. Acta Math. 111, 247–302 (1964)

    Article  MathSciNet  MATH  Google Scholar 

  49. Showalter, R.E.: Hilbert Space Methods for Partial Differential Equations. (Monographs and Studies in Mathematics.). Pitman, London (1977)

    MATH  Google Scholar 

  50. Showalter, R.E.: Monotone Operators in Banach Space and Nonlinear Partial Differential Equations. American Mathematical Society, Providence (1997)

    MATH  Google Scholar 

  51. Tröltzsch, F.: Optimal Control of Partial Differential Equations: Theory, Methods and Applications. Volume 112 of Graduate Studies in Mathematics. American Mathematical Society, Providence (2010). Translated from the 2005 German original by Jürgen Sprekels

    Book  Google Scholar 

  52. Zowe, J., Kurcyusz, S.: Regularity and stability for the mathematical programming problem in Banach spaces. Appl. Math. Optim. 5(1), 49–62 (1979)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Mathematics, Humboldt-University of Berlin, Unter den Linden 6, 10099, Berlin, Germany

    Michael Hintermüller, Carlos N. Rautenberg & Tao Wu

  2. Department of Mathematics, University of Stuttgart, Pfaffenwaldring 57/8.345, 70569, Stuttgart, Germany

    Andreas Langer

Authors
  1. Michael Hintermüller
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Carlos N. Rautenberg
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Tao Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Andreas Langer
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Michael Hintermüller.

Additional information

This research was carried out in the framework of Matheon supported by the Einstein Foundation Berlin within the ECMath projects OT1, SE5 and SE15 as well as by the DFG under Grant No. HI 1466/7-1 “Free Boundary Problems and Level Set Methods”.

A. Langer is listed as a co-author as he was involved in early numerical tests prior to writing this paper. In particular, he found the discretization of the \(\nabla \circ {\text {div}}\)-operator of [15] suitable for the present context, performed numerical tests concerning the choice of the upper level objective and the initial choice of \(\alpha =2.5~\times ~10^{-3}\) when solving the bilevel problem. He also provided the original source images used in Figs. 6 and 8.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hintermüller, M., Rautenberg, C.N., Wu, T. et al. Optimal Selection of the Regularization Function in a Weighted Total Variation Model. Part II: Algorithm, Its Analysis and Numerical Tests. J Math Imaging Vis 59, 515–533 (2017). https://doi.org/10.1007/s10851-017-0736-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10851-017-0736-2

Keywords

Mathematics Subject Classification

Search

Navigation