Skip to main content
SpringerLink
Log in

3D Face Reconstruction from 2D Pictures: First Results of a Web-Based Computer Aided System for Aesthetic Procedures

  • Published:
Annals of Biomedical Engineering Aims and scope Submit manuscript

Abstract

The human face is a vital component of our identity and many people undergo medical aesthetics procedures in order to achieve an ideal or desired look. However, communication between physician and patient is fundamental to understand the patient’s wishes and to achieve the desired results. To date, most plastic surgeons rely on either “free hand” 2D drawings on picture printouts or computerized picture morphing. Alternatively, hardware dependent solutions allow facial shapes to be created and planned in 3D, but they are usually expensive or complex to handle. To offer a simple and hardware independent solution, we propose a web-based application that uses 3 standard 2D pictures to create a 3D representation of the patient’s face on which facial aesthetic procedures such as filling, skin clearing or rejuvenation, and rhinoplasty are planned in 3D. The proposed application couples a set of well-established methods together in a novel manner to optimize 3D reconstructions for clinical use. Face reconstructions performed with the application were evaluated by two plastic surgeons and also compared to ground truth data. Results showed the application can provide accurate 3D face representations to be used in clinics (within an average of 2 mm error) in less than 5 min.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Figure 1
Figure 2
Figure 3
Figure 4
Figure 5
Figure 6
Figure 7
Figure 8

Similar content being viewed by others

References

  1. Besl, P. J., and H. D. McKay. A method for registration of 3-D shapes. IEEE Trans. Pattern Anal. Mach. Intell. 14:239–256, 1992.

    Article  Google Scholar 

  2. Blanz, V., A. Mehl, T. Vetter, and H. Seidel. A statistical method for robust 3D surface reconstruction from sparse data. In: International symposium on 3D data processing, visualization and transmission, pp. 293–300, 2004.

  3. Blanz, V., and T. Vetter. A morphable model for the synthesis of 3D faces. In: Proceedings of the 26th annual conference on computer graphics and interactive techniques, New York, NY, pp. 187–194, 1999.

  4. Burt, P. J., and E. H. Adelson. A multiresolution spline with application to image mosaics. ACM Trans. Graph. 2:217–236, 1983.

    Article  Google Scholar 

  5. Cootes, T. F., C. J. Taylor, D. H. Cooper, and J. Graham. Active shape models—their training and application. Comput. Vis. Image Underst. 61:38–59, 1995.

    Article  Google Scholar 

  6. Darisi, T., S. Thorne, and C. Iacobelli. Influences on decision-making for undergoing plastic surgery: a mental models and quantitative assessment. Plast. Reconstr. Surg. 116:907–916, 2005.

    Article  PubMed  CAS  Google Scholar 

  7. De Heras Ciechomski, P., M. Constantinescu, J. Garcia, R. Olariu, I. Dindoyal, S. Le Huu, and M. Reyes. Development and implementation of a web-enabled 3D consultation tool for breast augmentation surgery based on 3D-image reconstruction of 2D pictures. J. Med. Internet Res. 14:e21, 2012.

    Google Scholar 

  8. Faggian, N., A. Paplinski, and J. Sherrah. 3D morphable model fitting from multiple views. In: 8th IEEE international conference on automatic face & gesture recognition, 2008 (FG’08), Amsterdam, The Netherlands, pp. 1–6, 2008.

  9. Floater, M. S., and K. Hormann. Surface Parameterization: a Tutorial and Survey. In: Advances in Multiresolution for Geometric Modelling, edited by N. A. Dodgson, M. S. Floater, and M. A. Sabin. Springer Berlin Heidelberg, 2005, pp. 157–186.

  10. Flury, B., and H. Riedwyl. Multivariate Statistics: A Practical Approach. London: Chapman and Hall, 296 pp., 1988.

  11. Gower, J. Generalized procrustes analysis. Psychometrika 40:33–51, 1975.

    Article  Google Scholar 

  12. Heike, C. L., K. Upson, E. Stuhaug, and S. M. Weinberg. 3D digital stereophotogrammetry: a practical guide to facial image acquisition. Head Face Med. 6:18, 2010.

    Article  PubMed  Google Scholar 

  13. Jebara, T., A. Azarbayejani, and A. Pentland. 3D structure from 2D motion. IEEE Signal Process. Mag. 16:66–84, 1999.

    Article  Google Scholar 

  14. Kim, H., P. Jürgens, S. Weber, L.-P. Nolte, and M. Reyes. A new soft-tissue simulation strategy for cranio-maxillofacial surgery using facial muscle template model. Prog. Biophys. Mol. Biol. 103:284–291, 2010.

    Article  PubMed  Google Scholar 

  15. Koch, R. M., M. H. Gross, F. R. Carls, D. F. von Büren, G. Fankhauser, and Y. I. H. Parish. Simulating facial surgery using finite element models. In: Proceedings of the 23rd annual conference on computer graphics and interactive techniques, New York, NY, pp. 421–428, 1996.

  16. Lin, S. J., N. Patel, K. O’Shaughnessy, and N. Fine. A new three-dimensional imaging device in facial aesthetic and reconstructive surgery. Otolaryngol Head Neck Surg 139:313–315, 2008.

    Article  PubMed  Google Scholar 

  17. Milborrow, S., and F. Nicolls. Locating Facial features with an extended active shape model. In: Proceedings of the 10th European conference on computer vision: part IV, Berlin, Heidelberg, pp. 504–513, 2008.

  18. Moghaddam, B., J. Lee, H. Pfister, and R. Machiraju. Model-based 3D face capture with shape-from-silhouettes. In: IEEE international workshop on analysis and modeling of faces and gestures, 2003 (AMFG 2003), Nice, France, pp. 20–27, 2003.

  19. Ozkul, T., and M. H. Ozkul. Computer simulation tool for rhinoplasty planning. Comput. Biol. Med. 34:697–718, 2004.

    Article  PubMed  CAS  Google Scholar 

  20. Rabi, S. A., and P. Aarabi. Face fusion: an automatic method for virtual plastic surgery. In: 9th international conference on information fusion, Florence, Italy, pp. 1–7, 2006.

  21. Salzmann, M., J. Pilet, S. Ilic, and P. Fua. Surface deformation models for nonrigid 3D shape recovery. IEEE Trans. Pattern Anal. Mach. Intell. 29:1481–1487, 2007.

    Article  PubMed  Google Scholar 

  22. See, M., M. Foxton, N. Miedzianowski-Sinclair, C. Roberts, and C. Nduka. Stereophotogrammetric measurement of the nasolabial fold in repose: a study of age and posture-related changes. Eur. J. Plast. Surg. 29:387–393, 2007.

    Article  Google Scholar 

  23. Torresani, L., A. Hertzmann, and C. Bregler. Nonrigid structure-from-motion: estimating shape and motion with hierarchical priors. IEEE Trans. Pattern Anal. Mach. Intell. 30:878–892, 2008.

    Article  PubMed  Google Scholar 

  24. Zhang, R., P.-S. Tsai, J. E. Cryer, and M. Shah. Shape from shading: a survey. IEEE Trans. Pattern Anal. Mach. Intell. 21:690–706, 1999.

    Article  Google Scholar 

Download references

Acknowledgments

We acknowledge the support of the Swiss KTI Promotion Agency (grant: 12892.1 PFLS-LS).

Author information

Authors and Affiliations

  1. Institute for Surgical Technology and Biomechanics, University of Bern, Stauffacherstr. 78, 3014, Bern, Switzerland

    Thiago Oliveira-Santos, Christian Baumberger, Lutz-Peter Nolte, Salman Alaraibi & Mauricio Reyes

  2. Department of Plastic, Reconstructive, and Aesthetic Surgery, Inselspital, Bern, Switzerland

    Mihai Constantinescu & Radu Olariu

Authors
  1. Thiago Oliveira-Santos
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Christian Baumberger
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mihai Constantinescu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Radu Olariu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Lutz-Peter Nolte
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Salman Alaraibi
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Mauricio Reyes
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Thiago Oliveira-Santos.

Additional information

Associate Editor Ioannis A. Kakadiaris oversaw the review of this article.

Electronic supplementary material

Below is the link to the electronic supplementary material.

10439_2013_744_MOESM1_ESM.tif

Location of the 15 landmarks used for the TPS + CPM metric: rft and lft, right and left frontotemporale; rex and lex, right and left exocanthion; ren and len, right and left endocanthion; na, nasion; prn, pronasale; ral and lal, right and left, alare; sn, subnasale; ls, labiale superius; rch and lch, right and left cheilion; and gn, gnathion (TIFF 1283 kb)

Rights and permissions

Reprints and permissions

About this article

Cite this article

Oliveira-Santos, T., Baumberger, C., Constantinescu, M. et al. 3D Face Reconstruction from 2D Pictures: First Results of a Web-Based Computer Aided System for Aesthetic Procedures. Ann Biomed Eng 41, 952–966 (2013). https://doi.org/10.1007/s10439-013-0744-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10439-013-0744-3

Keywords

Search

Navigation