Skip to main content
SpringerLink
Log in

Fully automatic alpha matte extraction using artificial neural networks

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

The alpha matte is a two-dimensional map that is used to combine two images, one containing a foreground and the other containing a background. Alpha matte extraction is performed on green-screen images and requires user interaction to tune parameters in different preprocessing and postprocessing stages to refine an alpha matte. This paper tackles the problem of fully automatic extraction of the foreground on green-screen images with extraction of the corresponding alpha matte. The method is based on a multi-layer perceptron that assigns an alpha value, from a discrete set of ten alpha values, to each patch on a green-screen image. The approach for assigning an alpha value to an image patch is based on a set of features that enhance discrimination between foreground and background. The classifier is trained to learn to separate foreground objects from green-screen backgrounds as well as to generate the corresponding alpha matte map required for subsequent digital compositing. To test how the proposed approach handles alpha matte extraction under unsuitable conditions, a 64-image dataset was generated. The main contribution is that our method overcomes two challenges publicly posed within a dataset of green-screen image sequences, donated by Hollywood Camera Work LLC. Tests with this dataset generate high-quality visual results for those two cases. These results are confirmed by comparing the proposed fully automatic alpha matte extraction with that based on the use of Adobe After Effects Creative Cloud, an application which heavily depends on user interaction.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Chen Q, Li D, Tang CK (2013) KNN matting. IEEE Trans Pattern Anal Mach Intell 35(9):2175–2188

    Article  Google Scholar 

  2. Cao G, Li J, He Z, Chen X (2016) Divide and conquer: a self-adaptive approach for high-resolution image matting. In: Proceedings of international conference on virtual real and visualization, China. https://doi.org/10.1109/ICVRV.2016.13

  3. Phoka T, Jariyawattanarat W, Sudsang A (2017) Fine tuning for green-screen matting. In: Proceedings of IEEE international conference on knowledge and smart technology, Thailand. https://doi.org/10.1109/KST.2017.7886106

  4. Zhu X, Wang P, Huang Z (2017) Adaptive propagation matting based on transparency of image. Int J Multimed Tools Appl. https://doi.org/10.1007/s11042-017-5357-7

    Article  Google Scholar 

  5. Rosas-Romero R, López-Rincón O, Rojas-Velázquez ED, Jacobo-Aispuro NP (2016) Learning matte extraction in green-screen images with MLP classifiers and the back-propagation algorithm. In: Proceedings of 26th international conference on electronics, communication and computers, Mexico

  6. Gastal ES, Oliveira MM (2010) Shared sampling for realtime alpha matting. Comput Gr Forum 29:575–584

    Article  Google Scholar 

  7. Shahrian E, Rajan D, Price B, Cohen S (2013) Improving image matting using comprehensive sampling sets. In: Proceedings of IEEE conference on computer vision and pattern recognition, USA. https://doi.org/10.1109/CVPR.2013.88

  8. Al-Kabbany A, Dubois E (2016) Matting with sequential pair selection using graph transduction. In: Proceedings of international symposium on vision, model, and visualization, Germany. https://doi.org/10.2312/vmv.20161349

  9. Xu N, Price B, Cohen S, Huang T (2017) Deep image matting. https://arxiv.org/abs/1703.03872

  10. Wang J, Cohen MF (2008) Image and video matting: a survey. Found Trends Comput Gr Vis 3(2):97–175

    Article  Google Scholar 

  11. Chen X, He F (2016) A propagation matting method based on the local sampling and knn classification with adaptive feature space. J Comput Aided Des Comput Gr. https://arxiv.org/ftp/arxiv/papers/1605/1605.00732.pdf

  12. Ruzon MA, Tomasi C (2000) Alpha estimation in natural images. In: Proceedings of IEEE conference on computer vision and pattern recognition, USA. https://doi.org/10.1109/CVPR.2000.855793

  13. He K, Rhemann C, Rother C, Tang X, Sun J (2011) A global sampling method for alpha matting. In: Proceedings of IEEE conference on computer vision and pattern recognition, USA. https://doi.org/10.1109/CVPR.2011.5995495

  14. Chuang YY, Curless B, Salesin DH (2001) Bayesian approach to digital matting. In: Proceedings of IEEE conference on computer vision and pattern recognition, USA, vol 2, pp 264–271

  15. Rhemann C, Rother C, Gelautz M (2008) Improving color modeling for alpha matting. In: Proceedings of British machine conference. https://doi.org/10.5244/C.22.115

  16. Levin A, Rav AA (2008) Spectral matting. IEEE Trans Pattern Anal Mach Intell 30(10):1699–1712

    Article  Google Scholar 

  17. Wang J (2013) Image matting with transductive inference. Comput Vis Comput Gr Collab Technol 6930:239–250

    MathSciNet  Google Scholar 

  18. Johnson J, Rajan D, Cholakkal H (2014) Sparse codes as alpha mattes. In: Proceedings of British conference BMVC. https://doi.org/10.5244/C.28.74

  19. Wang J, Cohen MF (2005) An iterative optimization approach for unified image segmentation and matting. In: Proceedings of IEEE conference on computer vision, China. https://doi.org/10.1109/ICCV.2005.37

  20. Nath D, Chitra P (2015) Image matting based on weighted color and texture sample selection. Biomed Pharmacol J 8(1):331–335

    Article  Google Scholar 

  21. Yao G, Zhao Z, Liu S (2017) A comprehensive survey on sampling-based image matting. Comput Gr Forum 36:613–628

    Article  Google Scholar 

  22. Sun J, Jia J, Tang CK, Smith HY (2004) Poisson matting. J ACM Trans Gr 23(3):315–321

    Article  Google Scholar 

  23. Bai X, Sapiro G (2007) A geodesic framework for fast interactive image and video segmentation and matting. In: Proceedings of IEEE conference on computer vision, Brazil, pp 1–8

  24. He K, Sun J, Tang X (2010) Fast matting using large kernel Laplacian matrices. In: Proceedings of IEEE conference on computer vision and pattern recognition, USA. https://doi.org/10.1109/CVPR.2010.5539896

  25. Levin A, Lischinski D, Weiss Y (2007) A closed-form solution to natural image matting. IEEE Trans Pattern Anal Mach Intell 30(2):228–242

    Article  Google Scholar 

  26. Singaraju D, Rother C, Rhemann C (2009) New appearance models for natural image matting. In: Proceedings of IEEE conference on computer vision and pattern recognition, USA. https://doi.org/10.1109/CVPR.2009.5206491

  27. Shen X, Tao X, Gao H, Zhou C, Jia J (2016) Deep automatic portrait matting. In: Proceedings of European conference on computer vision, Netherlands. https://doi.org/10.1007/978-3-319-46448-0_6

    Chapter  Google Scholar 

  28. Cho D, Tai YW, Kweon I (2016) Natural image matting using deep convolutional neural networks. In: Leibe B, Matas J, Sebe N, Welling M (eds) Lecture notes in computer science, vol 9906. Springer, Berlin. https://doi.org/10.1007/978-3-319-46475-6_39

    Chapter  Google Scholar 

  29. Hollywood Camera Work LLC (2017) Green-screen plates. https://www.hollywoodcamerawork.com/green-screen-plates.html Accessed 13 Mar 2018

  30. Online benchmark dataset alpha matting (2009) https://www.alphamatting.com. Accessed 20 Feb 2019

  31. Wagner R, Thom M, Schweiger R, Rothermel A (2013) Learning convolutional neural networks from few samples. In: Proceedings of joint conference on neural network, USA (2013). https://doi.org/10.1109/IJCNN.2013.6706969

Download references

Acknowledgements

This research was partially supported by European Grant EU FP7 PEOPLE IRSES (CASES: Cloud computing-based Advisory Services towards Energy-efficient manufacturing Systems).

Author information

Authors and Affiliations

  1. Department of Electrical and Computer Engineering, Universidad de las Americas-Puebla, Sta. Catarina Martir, 72810, Cholula, Pue, Mexico

    Roberto Rosas-Romero, Omar Lopez-Rincon & Oleg Starostenko

Authors
  1. Roberto Rosas-Romero
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Omar Lopez-Rincon
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Oleg Starostenko
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Oleg Starostenko.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Rosas-Romero, R., Lopez-Rincon, O. & Starostenko, O. Fully automatic alpha matte extraction using artificial neural networks. Neural Comput & Applic 32, 6843–6855 (2020). https://doi.org/10.1007/s00521-019-04154-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-019-04154-4

Keywords

Search

Navigation