Skip to main content
SpringerLink
Log in

Image Sequence Analysis via Partial Differential Equations

  • Published:
Journal of Mathematical Imaging and Vision Aims and scope Submit manuscript

Abstract

This article deals with the problem of restoring and motion segmenting noisy image sequences with a static background. Usually, motion segmentation and image restoration are considered separately in image sequence restoration. Moreover, motion segmentation is often noise sensitive. In this article, the motion segmentation and the image restoration parts are performed in a coupled way, allowing the motion segmentation part to positively influence the restoration part and vice-versa. This is the key of our approach that allows to deal simultaneously with the problem of restoration and motion segmentation. To this end, we propose a theoretically justified optimization problem that permits to take into account both requirements. The model is theoretically justified. Existence and unicity are proved in the space of bounded variations. A suitable numerical scheme based on half quadratic minimization is then proposed and its convergence and stability demonstrated. Experimental results obtained on noisy synthetic data and real images will illustrate the capabilities of this original and promising approach.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. T. Aach and A. Kaup, “Bayesian algorithms for adaptive change detection in image sequences using markov random fields,” Signal Processing: Image Communication, Vol. 7, pp. 147–160, 1995.

    Google Scholar 

  2. R. Acart and C.R. Vogel, “Analysis of bounded variation penalty methods for ill-posed problems,” Inverse Problems, Vol. 10, pp. 1217–1229, 1994.

    Google Scholar 

  3. L. Alvarez, P.-L. Lions, and J.-M. Morel, “Image selective smoothing and edge detection by nonlinear diffusion,” SIAM Journal of Numerical Analysis, Vol. 29, pp. 845–866, 1992.

    Google Scholar 

  4. L. Alvarez and L. Mazorra, “Signal and image restoration using shock filters and anisotropic diffusion,” SIAM Journal of Numerical Analysis, Vol. 31, No. 2, pp. 590–605, 1994.

    Google Scholar 

  5. L. Ambrosio, “A compactness theorem for a new class of functions of bounded variation,” Boll. Unione Mat. Ital.,Vol. 7, No. 4, pp. 857–881, 1989.

    Google Scholar 

  6. L. Ambrosio and G. Dal Maso, “A general chain rule for distributional derivatives,” Proc. Am. Math. Soc., Vol. 108, No. 3, pp. 691–702, 1990.

    Google Scholar 

  7. G. Anzellotti, “The euler equation for functionals with linear growth,” Trans. Am. Math. Soc., Vol. 290, No. 2, pp. 483–501, 1985.

    Google Scholar 

  8. G. Aubert, M. Barlaud, L. Blanc-Feraud, and P. Charbonnier, “Deterministic edge-preserving regularization in computed imaging,” IEEE Trans. Imag. Process., Vol. 5, No. 12, 1997.

  9. G. Aubert, R. Deriche, and P. Kornprobst, “Amathematical study of the regularized optical flow problem in the space BV (Ω),” Technical Report 503, Université de Nice-Sophia Antipolis, Dec. 1997.

  10. G. Aubert, R. Deriche, and P. Kornprobst, “A variational method and its mathematical study in image sequence analysis,” Technical Report 3415, INRIA, April 1998.

  11. G. Aubert and L. Vese, “Avariational method in image recovery,” SIAM J. Numer. Anal., Vol. 34, No. 5, pp. 1948–1979, 1997.

    Google Scholar 

  12. M.J. Black, G. Sapiro, D.H. Marimont, and D. Heeger, “Robust anisotropic diffusion,” IEEE Trans. Imag. Proc., Vol. 7, No. 3, pp. 421–432, 1998. Special Issue on Partial Differential Equations and Geometry-Driven Diffusion in Image Processing and Analysis.

    Google Scholar 

  13. A. Blake and A. Zisserman, Visual Reconstruction, MIT Press, 1987.

  14. P. Blomgren and T.F. Chan, “Color Tv: Total variation methods for restoration of vector-valued images,” IEEE Trans. Imag. Proc., Vol. 7, No. 3, pp. 304–309, 1998. Special Issue on Partial Differential Equations and Geometry-Driven Diffusion in Image Processing and Analysis.

    Google Scholar 

  15. G. Bouchitté, I. Fonseca, and L. Mascarenhas, “A global method for relaxation,” Technical report, Université de Toulon et du Var, May 1997.

  16. J. Boyce, “Noise reduction of image sequences using adaptative motion compensated frame averaging,” in IEEE ICASSP, 1992, Vol. 3, pp. 461–464.

    Google Scholar 

  17. J.C. Brailean and A.K. Katsaggelos, “Simultaneous recursive displacement estimation and restoration of noisy-blurred image sequences,” IEEE Transactions on Image Processing, Vol. 4, No. 9, pp. 1236–1251, 1995.

    Google Scholar 

  18. H. Brezis, Opérateurs maximaux monotones et semi-groupes de contractions dans les espaces de Hilbert, North-Holland Publishing Comp: Amsterdam-London, 1973.

    Google Scholar 

  19. O. Buisson, B. Besserer, S. Boukir, and F. Helt, “Deterioration detection for digital film restoration,” in Computer Vision and Pattern Recognition, Puerto Rico, June 1997, pp. 78–84.

  20. V. Caselles, J.M. Morel, G. Sapiro, and A. Tannenbaum, “Introduction to the special issue on partial differential equations and geometry-driven diffusion in image processing and analysis,” IEEE Transactions on Image Processing, Vol. 7, No. 3, pp. 269–273, 1998.

    Google Scholar 

  21. A. Chambolle and P.-L. Lions, “Image recovery via total variation minimization and related problems,” Numer. Math., Vol. 76, No. 2, pp. 167–188, 1997.

    Google Scholar 

  22. I. Cohen, “Nonlinear variational method for optical flow computation,” in Proceedings 8th SCIA, 1993, Vol. 1, pp. 523–530.

    Google Scholar 

  23. L.D. Cohen, Auxiliary variables and two-step iterative algorithms in computer vision problems, ICCV, 1995.

  24. G.-H. Cottet and L. Germain, “Image processing through reaction combined with nonlinear diffusion,” Mathematics of Computation, Vol. 61, No. 204, pp. 659–673, 1993.

    Google Scholar 

  25. F. Demengel and R. Temam, “Convex functions of a measure and applications,” Indiana University Mathematics Journal, Vol. 33, pp. 673–709, 1984.

    Google Scholar 

  26. R. Deriche and O. Faugeras, “Les EDP en traitement des images et vision par ordinateur,” Technical report, INRIA, Nov. 1995. A more complete version of this Research Report has appeared in the French Revue, Traitement du Signal, Vol. 13, No. 6, 1996 (special issue).

  27. E. Dubois and S. Sabri, “Noise reduction in image sequences using motion-compensated temporal filtering,” IEEE Transactions on Communications, Vol. 32, No. 7, pp. 826–831, 1984.

    Google Scholar 

  28. L.C. Evans and R.F. Gariepy, Measure Theory and Fine Properties of Functions, CRC, 1992.

  29. H. Federer, Geometric Measure Theory, “Classics in Mathematics,” Springer-Verlag, Berlin, Heidelberg, New York, 1969.

    Google Scholar 

  30. D. Geman and G. Reynolds, “Constrained restoration and the recovery of discontinuities,” IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 14, No. 3, pp. 367–383, 1993.

    Google Scholar 

  31. S. Geman, D.E. McClure, and D. Geman, “A nonlinear filter for film restoration and other problems in image processing,” CVGIP: Graphical Models and Image Processing, Vol. 54, No. 4, pp. 281–289, 1992.

    Google Scholar 

  32. E. De Giorgi and T. Franzoni, “Su un tipo di convergenza variazionale,” Atti Accad. Naz. Lincei Rend. Cl. Sci. Fis. Mat. Natur., Vol. 68, pp. 842–850, 1975.

    Google Scholar 

  33. E. Giusti, Minimal Surfaces and Functions of Bounded Variation, Birkhäuser, 1984.

  34. C. Goffman and J. Serrin, “Sublinear functions of measures and variational integrals,” Duke Math J., Vol. 31, pp. 159–178, 1964.

    Google Scholar 

  35. F. Guichard, “Axiomatisation des analyses multi-échelles d'images et de films,” Ph.D. thesis, Université Paris IX Dauphine, 1994.

  36. K. Karmann, A. Brandt, and R. Gerl, “Moving object segmentation based on adaptive reference images,” Signal Processing: Theories and Applications, Vol. 5, pp. 951–954, 1990.

    Google Scholar 

  37. D. Kinderlehrer and G. Stampacchia, An Introduction to Variational Inequalities and Their Applications, Academic Press, 1980.

  38. A. Kokaram, “Reconstruction of severely degraded image sequences,” in International Conference on Image Applications and Processing, Florence, Italy, 1997.

  39. A.C. Kokaram and S.J. Godsill, “A system for reconstruction of missing data in image sequences using sampled 3D SAR models and mrf motion priors,” in Proceedings of the 4th European Conference on Computer Vision, Bernard Buxton (Ed.), Cambridge, UK, April 1996, pp. 613–624.

  40. P. Kornprobst, “Contributions à la restauration d'images et à l'analyse de séquences: Approches Variationnelles et Equations aux Dérivées Partielles,” Ph.D. thesis, Université de Nice-Sophia Antipolis, 1998.

  41. P. Kornprobst, R. Deriche, and G. Aubert, “Image restoration via PDE',” in First Annual Symposium on Enabling Technologies for Law Enforcement and Security-SPIE Conference 2942: Investigative Image Processing, Boston, Massachusetts, USA., Nov. 1996.

  42. P. Kornprobst, R. Deriche, and G. Aubert, “Image coupling, restoration and enhancement via PDE',” in International Conference on Image Processing, vol. II of III, Santa-Barbara, California, Oct. 1997, pp. 458–461.

    Google Scholar 

  43. P. Kornprobst, R. Deriche, and G. Aubert, “Nonlinear operators in image restoration,” in Proceedings of the International Conference on Computer Vision and Pattern Recognition, Puerto-Rico, IEEE, June 1997, pp. 325–331.

    Google Scholar 

  44. P. Kornprobst, R. Deriche, and G. Aubert, “Edp, débruitage et réhaussement en traitement d'image: Analyse et contributions,” in 11 ème Congres RFIA, AFCET, Jan. 1998.

  45. S. Liou and R. Jain, “Motion detection in spatio-temporal space,” Computer Vision, Graphics and Image Understanding, Vol. 45, pp. 227–250, 1989.

    Google Scholar 

  46. R. Malladi and J.A. Sethian, “Image processing: Flows under min/max curvature and mean curvature,” Graphical Models and Image Processing, Vol. 58, No. 2, pp. 127–141, March 1996.

    Google Scholar 

  47. G. Dal Maso, An introduction to Г-convergence, Progress in Nonlinear Differential Equations and their Applications, Birkhauser, 1993.

  48. N.G. Meyers, “An l p-estimate for the gradient of solutions of second order elliptic divergence equations,” Ann. Scuola Norm. Sup. Pisa, Vol. 3, No. 17, pp. 189–206, 1963.

    Google Scholar 

  49. N.G. Meyers and A. Elcrat, “Some results on regularity for solutions of non-linear elliptic systems and quasi-regular functions,” Duke math. J., Vol. 42, pp. 121–136, 1975.

    Google Scholar 

  50. L. Moisan, “Traitement numérique d'images et de films: équations aux dérivées partielles préservant forme et relief,” Ph.D. thesis, Université Paris IX Dauphine, June 1997.

  51. J.-M. Morel and S. Solimini, “Segmentation of images by variational methods: A constructive approach,” Rev. Math. Univ. Complut. Madrid, Vol. 1, pp. 169–182, 1988.

    Google Scholar 

  52. R.D. Morris, “Image Sequence Restoration using Gibbs Distributions,” Ph.D. thesis, Cambridge University, England, 1995.

    Google Scholar 

  53. D. Mumford and J. Shah, “Optimal approximations by piecewise smooth functions and associated variational problems,” Comm. Pure Appl. Math., Vol. 42, pp. 577–684, 1989.

    Google Scholar 

  54. N. Nordström, “sed anisotropic diffusion-A unified regularization and diffusion approach to edge detection” Image and Vision Computing, Vol. 8, No. 11, pp. 318–327, 1990.

    Google Scholar 

  55. N. Paragios and R. Deriche, “A PDE-based level set approach for detection and tracking of moving objects,” in Proceedings of the 6th International Conference on Computer Vision, Bombay, India, Jan. 1998. IEEE Computer Society Press.

  56. N. Paragios and G. Tziritas, “Detection and localization of moving objects in image sequences,” FORT-Hellas Technical Report, Accepted for publication in Signal Processing: Image Communication, Oct. 1996.

  57. N. Paragios and R. Deriche, “Detecting multiple moving targets using deformable contours,” in International Conference on Image Processing, Vol. II of III, Santa-Barbara, California, Oct. 1997, pp. 183–186.

    Google Scholar 

  58. P. Perona and J. Malik, “Scale-space and edge detection using anisotropic diffusion,” IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 12, No. 7, pp. 629–639, 1990.

    Google Scholar 

  59. M. Proesmans, E. Pauwels, and L. Van Gool, “Coupled geometry-driven diffusion equations for low-level vision,” Computational Imaging and Vision, Kluwer Academic Publishers, 1994, pp. 191–228.

  60. L. Rudin and S. Osher, “Total variation based image restoration with free local constraints,” in International Conference on Image Processing, Nov. 1994, Vol. 1, pp. 31–35.

    Google Scholar 

  61. G. Sapiro and V. Caselles, “Contrast enhancement via image evolution flows,” Graphical Models and Image Processing, Vol. 59, No. 6, pp. 407–416, 1997.

    Google Scholar 

  62. G. Sapiro and D.L. Ringach, “Anisotropic diffusion of multivalued images with applications to color filtering,” IEEE Transactions on Image Processing, Vol. 5, No. 11, pp. 1582–1585, 1996.

    Google Scholar 

  63. G. Sapiro, A. Tannenbaum, Y.L. You, and M. Kaveh, “Experiments on geometric image enhancement,” in International Conference on Image Processing, 1994.

  64. C. Schnörr, “Unique reconstruction of piecewise-smooth images by minimizing strictly convex nonquadratic functionals,” Journal of Mathematical Imaging and Vision, Vol. 4, pp. 189–198, 1994.

    Google Scholar 

  65. J. Shah, “A common framework for curve evolution, segmentation and anisotropic diffusion,” IEEE, 1996.

  66. R.L. Stevenson, B.E. Schmitz, and E.J. Delp, “Discontinuity preserving regularization of inverse visual problems,” IEEE Transactions on Systems, Man, and Cybernetics, Vol. 24, No. 3, pp. 455–469, 1994.

    Google Scholar 

  67. D.M. Strong and T.F. Chan, “Spatially and scale adaptive total variation based regularization and anisotropic diffusion in image processing,” Technical Report 46, UCLA, Nov. 1996.

  68. A.N. Tikhonov and V.Y. Arsenin, Solutions of Ill-posed Problems, Winston and Sons, Washington, DC, 1977.

    Google Scholar 

  69. L. Vese, “Problèmes variationnels et EDP pour l'analyse d'images et l'évolution de courbes,” Ph.D. thesis, Université de Nice Sophia-Antipolis, Nov. 1996.

  70. A.I. Vol'pert, “The spaces BV and quasilinear equations,” Math. USSR-Sbornik, Vol. 2, No. 2, pp. 225–267, 1967.

    Google Scholar 

  71. J. Weickert, “Anisotropic diffusion in image processing,” Ph.D. thesis, University of Kaiserslautern, Germany, Laboratory of Technomathematics, Jan. 1996.

    Google Scholar 

  72. J. Weickert, Anisotropic Diffusion in Image Processing, Teubner-Verlag, Stuttgart, 1998.

    Google Scholar 

  73. O. Wenstop, “Motion detection from image information,” Proceedings in Scandianvian Conference on Image Analysis, 1983, pp. 381–386.

  74. Y.L. You, M. Kaveh, W.Y. Xu, and A. Tannenbaum, “Analysis and design of anisotropic diffusion for image processing,” in International Conference on Image Processing, 1994, Vol. 2, pp. 497–501.

    Google Scholar 

  75. W.P. Ziemer, Weakly Differentiable Functions, Springer-Verlag, 1989.

Download references

Author information

Authors and Affiliations

  1. INRIA, 2004 route des Lucioles, BP 93, 06902, Sophia-Antipolis Cedex, France

    Pierre Kornprobst & Rachid Deriche

  2. Laboratoire J.A Dieudonne, UMR no 6621 du CNRS, 06108, Nice-Cedex 2, France

    Gilles Aubert

Authors
  1. Pierre Kornprobst
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Rachid Deriche
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Gilles Aubert
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Kornprobst, P., Deriche, R. & Aubert, G. Image Sequence Analysis via Partial Differential Equations. Journal of Mathematical Imaging and Vision 11, 5–26 (1999). https://doi.org/10.1023/A:1008318126505

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/A:1008318126505

Search

Navigation