Skip to main content
SpringerLink
Log in

Geodesic Matting: A Framework for Fast Interactive Image and Video Segmentation and Matting

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

An interactive framework for soft segmentation and matting of natural images and videos is presented in this paper. The proposed technique is based on the optimal, linear time, computation of weighted geodesic distances to user-provided scribbles, from which the whole data is automatically segmented. The weights are based on spatial and/or temporal gradients, considering the statistics of the pixels scribbled by the user, without explicit optical flow or any advanced and often computationally expensive feature detectors. These could be naturally added to the proposed framework as well if desired, in the form of weights in the geodesic distances. An automatic localized refinement step follows this fast segmentation in order to further improve the results and accurately compute the corresponding matte function. Additional constraints into the distance definition permit to efficiently handle occlusions such as people or objects crossing each other in a video sequence. The presentation of the framework is complemented with numerous and diverse examples, including extraction of moving foreground from dynamic background in video, natural and 3D medical images, and comparisons with the recent literature.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • ADOBE SYSTEMS INCORP. (2002). Adobe photoshop user guide.

  • ADOBE SYSTEMS INCORP. (2007). Adobe photoshop CS3 new features. http://www.adobe.com/products/photoshop/photoshop.

  • Agarwala, A., Hertzmann, A., Salesin, D., & Seitz, S. (2004). Keyframe-based tracking for rotoscoping and animation. In Proceedings of SIGGRAPH’04.

  • Bai, X., & Sapiro, G. (2007). A geodesic framework for fast interactive image and video segmentation and matting. In Proc. international conference computer vision, Rio de Janeiro, Brazil, 16–19 October 2007.

  • Black, M. J., & Anandan, P. (1996). The robust estimation of multiple motions: parametric and piecewise-smooth flow fields. Computer Vision and Image Understanding, 63(1), 75–104.

    Article  Google Scholar 

  • Boykov, Y. Y., & Jolly, M.-P. (2001). Interactive graph cuts for optimal boundary and region segmentation of objects in n-d images. In IEEE ICCV 2001 (Vol. 01, pp. 105).

  • Caselles, V., Morel, J.-M., & Sbert, C. (1998). An axiomatic approach to image interpolation. IEEE Transactions on Image Processing, 7(3), 376–386.

    Article  MATH  MathSciNet  Google Scholar 

  • Chuang, Y.-Y., Curless, B., Salesin, D. H., & Szeliski, R. (2001). A Bayesian approach to digital matting. In Proceedings of IEEE CVPR 2001 (Vol. 2, pp. 264–271). December 2001.

  • Chuang, Y.-Y., Agarwala, A., Curless, B., Salesin, D. H., & Szeliski, R. (2002). Video matting of complex scenes. In SIGGRAPH ’02 (pp. 243–248).

  • COREL CORPORATION. (2002). Knockout user guide.

  • Criminisi, A., Cross, G., Blake, A., & Kolmogorov, V. (2006). Bilayer segmentation of live video. In Proceedings of IEEE CVPR 2006 (pp. 53–60).

  • Dial, R. (1969). Shortest path forest with topological ordering. Communications of the ACM, 12, 632–633.

    Article  Google Scholar 

  • Falcao, A. X., Stolfi, J., & de Alencar Lotufo, R. (2004). The image foresting transform: Theory, algorithms, and applications. IEEE Transactions on Pattern Analysis and Machine Intelligence, 26(1), 19–29.

    Article  Google Scholar 

  • Grady, L. (2006). Random walks for image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(11), 1768–1783.

    Article  Google Scholar 

  • Grady, L., Schiwietz, T., Aharon, S., & Westermann, R. (2005). Random walks for interactive alpha-matting. In Proceedings of VIIP 2005 (pp. 423–429). Spain, September 2005. ACTA Press.

  • Levin, A., Lischinski, D., & Weiss, Y. (2004). Colorization using optimization. SIGGRAPH’04, 23(3), 689–694.

    Google Scholar 

  • Levin, A., Lischinski, D., & Weiss, Y. (2006). A closed form solution to natural image matting. In Proceedings of IEEE CVPR 2006 (pp. 61–68).

  • Levin, A., Rav-Acha, A., & Lischinski, D. (2007). Spectral matting. In Proceedings of IEEE CVPR 2007, June 2007.

  • Lowe, D. (2004). Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision, 2, 91–110.

    Article  Google Scholar 

  • Protiere, A., & Sapiro, G. (2007). Interactive image segmentation via adaptive weighted distances. IEEE Transactions on Image Processing, 16, 1046–1057.

    Article  MathSciNet  Google Scholar 

  • Rother, C., Kolmogorov, V., & Blake, A. (2004). Grabcut: Interactive foreground extraction using iterated graph cuts. In SIGGRAPH’04.

  • Sapiro, G. (2001). Geometric partial differential equations and image processing. Cambridge: Cambridge University Press.

    Google Scholar 

  • Sinop, A. K., & Grady, L. (2007). A seeded image segmentation framework unifying graph cuts and random walker which yields a new algorithm. In Proc. international conference computer vision, Rio de Janeiro, Brazil, 16–19 October 2007.

  • Sun, J., Jia, J., Tang, C.-K., & Shum, H.-Y. (2004). Poisson matting. In SIGGRAPH’04 (pp. 315–321).

  • Sun, J., Kang, S. B., Xu, Z., Tang, X., & Shum, H.-Y. (2007). Flash cut: Foreground extraction with flash and no-flash image pairs. In Proceedings of IEEE CVPR 2007, June 2007.

  • Thorup, M. (1997). Undirected single source shortest paths in linear time. In Proceedings IEEE symposium on foundations of computer science (pp. 12–21).

  • Wang, J., & Cohen, M. F. (2005). An iterative optimization approach for unified image segmentation and matting. In Proceedings of IEEE ICCV 2005 (pp. 936–943).

  • Wang, J., & Cohen, M. (2007a). Image and video matting: A survey. Foundations and Trends in Computer Graphics and Vision, 3(2), 1–78.

    MATH  Google Scholar 

  • Wang, J., & Cohen, M. (2007b). Optimized color sampling for robust matting. In Proceedings of IEEE CVPR 2007, June 2007.

  • Wang, J., Bhat, P., Colburn, R. A., Agrawala, M., & Cohen, M. F. (2005). Interactive video cutout. SIGGRAPH’05, 24(3), 585–594.

    Google Scholar 

  • Wang, J., Agrawala, M., & Cohen, M. F. (2007). Soft scissors: An interactive tool for realtime high quality matting. In SIGGRAPH’07.

  • Yang, C., Duraiswami, R., Gumerov, N., & Davis, L. (2003). Improved fast Gauss transform and efficient kernel density estimation. In Proceedings IEEE ICCV 2003 (pp. 464–471). Nice, France.

  • Yatziv, L., & Sapiro, G. (2005). Image and video data blending using intrinsic distances. Patent pending.

  • Yatziv, L., & Sapiro, G. (2006). Fast image and video colorization using chrominance blending. IEEE Transactions on Image Processing, 15(5), 1120–1129.

    Article  Google Scholar 

  • Yatziv, L., Bartesaghi, A., & Sapiro, G. (2006). O(n) implementation of the fast marching algorithm. Journal of Computational Physics, 212, 393–399.

    Article  MATH  Google Scholar 

  • Virtual colonoscopy screening resource center. (2005) http://www.vcscreen.com.

Download references

Author information

Authors and Affiliations

  1. Electrical and Computer Engineering, University of Minnesota, Minneapolis, MN, 55455, USA

    Xue Bai & Guillermo Sapiro

Authors
  1. Xue Bai
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Guillermo Sapiro
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Guillermo Sapiro.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Bai, X., Sapiro, G. Geodesic Matting: A Framework for Fast Interactive Image and Video Segmentation and Matting. Int J Comput Vis 82, 113–132 (2009). https://doi.org/10.1007/s11263-008-0191-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11263-008-0191-z

Keywords

Search

Navigation