Skip to main content
SpringerLink
Log in

3D geometry-based face recognition in presence of eye and mouth occlusions

  • Original Paper
  • Published:
International Journal on Interactive Design and Manufacturing (IJIDeM) Aims and scope Submit manuscript

Abstract

This study proposes a novel occlusions detection and restoration strategy. The aim is to success with 3D face recognition even when faces are partially occluded by external objects. The method, which relies on geometrical facial properties, is designed for managing two types of facial occlusions (eye and mouth occlusions due to hands). First occlusions are detected and (if present) classified, by considering their effects on the 3D points cloud. Then, the occluded regions are progressively removed, and finally, the non-occluded symmetrical regions are used to restore the missing information. After the restoration process, face recognition is performed relying on the restored facial information and on the localized landmarks. The landmarking methodology relies on derivatives and on 12 differential geometry descriptors. The discriminating features adopted for facial comparison include shape index histograms, Euclidean and geodetical distances between landmarks, facial curves, and nose volume. Obtained recognition rates, evaluated on the whole Bosphorus database and on our private dataset, ranging from 92.55 to 97.20% depending on the completeness of data.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19

Similar content being viewed by others

References

  1. Patil, H., Kothari, A., Bhurchandi, K.: 3-D face recognition: features, databases, algorithms and challenges. Artif. Intell. Rev. 44(3), 393–441 (2015)

    Article  Google Scholar 

  2. Szeptycki, P., Ardabilian M., Chen, L.: Nose tip localization on 2.5 D facial models using differential geometry based point signatures and SVM classifier. In: BIOSIG-Proceedings of the International Conference of the Biometrics Special Interest Group (BIOSIG), pp. 1–12 (2012)

  3. Kakadiaris, I.: A third dimension in face recognition (2012). https://doi.org/10.1117/2.1201205.004214

  4. Abate, A.F., Nappi, M., Riccio, D., Sabatino, G.: 2D and 3D face recognition: a survey. Pattern Recognit. Lett. 28(14), 1885–1906 (2007)

    Article  Google Scholar 

  5. Savvides, M., Abiantun, R., Heo, J., Park, S., Xie, C., Vijayakumar, B.: Machine, partial & holistic face recognition on FRGC-II data using support vector. In: Proceedings of the 2006 Conference on Computer Vision and Pattern Recognition Workshop (CVPRW’06) (2006)

  6. Drira, H., Amor, B.B., Srivastava, A., Daoudi, M., Slama, R.: 3D face recognition under expressions, occlusions and pose variations. IEEE Trans. Pattern Anal. Mach. Intell. 35(9), 2270–2283 (2013)

    Article  Google Scholar 

  7. Bhave, D., Choudhary, R., Gavali, R., Gholap, P.: 3D face recognition under expressions, occlusions, and pose variations. Int. J. Adv. Res. Comput. Commun. Eng. 5(3), 147–150 (2016)

    Google Scholar 

  8. Yu, X., Gao, Y., and Zhou, J.: 3D face recognition under partial occlusions using radial strings. In: IEEE International Conference on Image Processing (ICIP) (2016)

  9. Yu, X., Gao, Y., Zhou, J.: Boosting radial string for 3D face recognition with expressions and occlusions. In: International Conference on Digital Image Computing: Techniques and Applications (DICTA) (2016)

  10. Li, X., Da, F.: Efficient 3D face recognition handling facial expression and hair occlusion. Image Vis. Comput. 30(9), 668–679 (2012)

    Article  Google Scholar 

  11. Alyuz, N., Gokberk, B., Akarun, L.: A 3D face recognition system for expression and occlusion invariance. In: Biometrics: Theory, Applications and Systems, vol. 29 (2008)

  12. Alyuz, N., Gokberk, B., Akarun, L.: Adaptive registration for occlusion robust 3D face recognition. In: European Conference on Computer Vision, vol. 7585 (2012)

    Chapter  Google Scholar 

  13. Alyuz, N., Gokberk, B., Spreeuwers, L., Veldhuis, R., Akarun, L.: Robust 3D face recognition in the presence of realistic occlusions. In: 5th IAPR International Conference on Biometrics (ICB) (2012)

  14. Bagchi, P., Bhattacharjee, D., Nasipuri, M.: Robust 3D face recognition in presence of pose and partial occlusions or missing parts. Int. J. Found. Comput. Sci. Technol. (IJFCST) 4(4), 21–35 (2014)

    Article  Google Scholar 

  15. Ganguly, S., Bhattacharjee, D., Nasipuri, M.: Depth based occlusion detection and localization from 3D face image. Int. J. Image Graph. Signal Process. 5, 20–31 (2015)

    Article  Google Scholar 

  16. Zohra, F.T., Rahman, W., Gavrilova, M.: Occlusion detection and localization from Kinect depth images. In: International Conference on Cyberworlds (2016)

  17. Min, R., Kose, N., Dugelay, J.: KinectFaceDB: a kinect database for face recognition. IEEE Trans. Syst. Man Cybern. 44(11), 1534–1548 (2014)

    Article  Google Scholar 

  18. Tsapatsoulis, N., Doulamis, N., Doulamis, A., Kollias, S.: Face extraction from non-uniform background and recognition in compressed domain. In: Proceedings of the IEEE International Conference on Acoustics, Speech and Signal, vol. 5, pp. 2701–2704 (1998)

  19. Chang, K.I., Bowyer, K.W., Flynn, P.J.: An evaluation of multimodal 2D + 3D face biometrics. IEEE Trans. Pattern Anal. Mach. Intell. 27, 619–624 (2005)

    Article  Google Scholar 

  20. Samani, A., Winkler, J., Niranjan, M.: Automatic face recognition using stereo images. In: IEEE International Conference on Acoustics, Speech and Signal Processing (2006)

  21. Colombo, A., Cusano, C., Schettin, R.: Gappy PCA classification for occlusion tolerant 3D face detection. J. Math. Imaging Vis. 35, 193 (2009)

    Article  MathSciNet  Google Scholar 

  22. Alyuz, N., Gokberk, B., Akarun, L.: Detection of realistic facial occlusions for robust 3D face recognition. In: 22nd International Conference on Pattern Recognition (2014)

  23. Srinivasan, A., Balamurugan, V.: Occlusion detection and image restoration in 3D face image. In: IEEE Region Conference TENCON (2014)

  24. Liu, P., Wang, Y., Huang, D., Zhaoxiang, Z.: Recognizing occluded 3D faces using an efficient ICP variant. In: IEEE International Conference on Multimedia and Expo (2012)

  25. Colombo, A., Cusano, C., Schettini, R.: Detection and restoration of occlusions for 3D face recognition. In: IEEE International Conference on Multimedia and Expo, pp. 1541–1544 (2006)

  26. Everson, R., Sirovich, L.: Karhunen–Loeve procedure for gappy data. J. Opt. Soc. Am. A 12(8), 1657–1664 (1995)

    Article  Google Scholar 

  27. Alyuz, N., Gokberk, B., Akarun, L.: 3-D face recognition under occlusion using masked projection. IEEE Trans. Inf. Forensics Secur. 8(5), 789–802 (2013)

    Article  Google Scholar 

  28. Drira, H., Ben Amor, B., Srivastava, A., Daoudi, M., Slama, R.: 3D face recognition under expressions, occlusions and pose variations. In: IEEE Transactions on Pattern Analysis and Machine Intelligence, Institute of Electrical and Electronics Engineers, , pp. 2270–2283 (2013)

    Article  Google Scholar 

  29. Drira, H., Ben Amor, B., Srivastava, A., Daoudi, M., Slama, R.: 3D face recognition using geodesic facial curves to handle expression, occlusions and pose variations. Int. J. Comput. Sci. Inf. Technol. (IJCSIT) 53, 4284–4287 (2014)

    Google Scholar 

  30. Gawali, S., Deshmukh, R.R.: 3D face recognition using geodesic facial curves to handle expression, occlusion and pose variations. Int. J. Comput. Sci. Inf. Technol. 5(3), 4284–4287 (2014)

    Google Scholar 

  31. Xiaoli Li, F.D.: Efficient 3D face recognition handling facial expression and hair occlusion. Image Vis. Comput. 30, 668–679 (2012)

    Article  Google Scholar 

  32. Vezzetti, E., Marcolin, F., Tornincasa, S., Dagnes, N.: 3D geometry-based automatic landmark localization in presence of facial occlusions. Multimed. Tools Appl. (under review)

  33. Penke, L., Bates, T.C., Gow, A.J., Pattie, A., Starr, J.M., Jones, B.C., Perrett, D.I.: Symmetric faces are a sign of successful cognitive aging. Evolut. Hum. Behav. 30(6), 429–437 (2009)

    Article  Google Scholar 

  34. Liu, Y., Palmer, J.: A quantified study of facial asymmetry in 3D faces. In: Proceedings of the IEEE International Workshop on Analysis and Modeling of Faces and Gestures. IEEE Computer Society (2003)

  35. Vezzetti, E., Marcolin, F., Tornincasa, S., Ulrich, L., Dagnes, N.: 3D geometry-based automatic landmark localization in presence of facial occlusions. Multimed. Tools Appl. 77(11), 14177–14205 (2018)

    Article  Google Scholar 

  36. Koenderink, J.J., van Doorn, A.J.: Surface shape and curvature scales. Image Vis. Comput. 10(8), 557–564 (1992)

    Article  Google Scholar 

  37. Vezzetti, E., Marcolin, F.: Geometrical descriptors for human face morphological analysis and recognition. Robot. Auton. Syst. 60(6), 928–939 (2012)

    Article  Google Scholar 

  38. Cirrincione, G., Marcolin, F., Spada, S., Vezzetti, E.: Intelligent quality assessment of geometrical features for 3D face recognition. In: 27th Italian Workshop on Neural Networks (WIRN), Vietri sul Mare, Salerno, Italy (2017)

  39. Zhan, C., Li, W., Ogunbona, P.: Face recognition from single sample based on human face perception. In: 24th International Conference in Image and Vision Computing New Zealand (IVCNZ’09), New Zealand (2009)

  40. Vezzetti, E., Marcolin, F., Fracastoro, G.: 3D face recognition: an automatic strategy based on geometrical descriptors and landmarks. Robot. Autonom. Syst. 62(12), 1768–1776 (2014)

    Article  Google Scholar 

  41. Bellil, W., Brahim, H., Ben Amar, C.: Gappy wavelet neural network for 3D occluded faces: detection and recognition. Multimed. Tools Appl. 75, 36–380 (2016)

    Article  Google Scholar 

  42. Li, H., Huang, D., Lemaire, P., Morvan, J., Chen, L.: Expression robust 3D face recognition via mesh-based histograms of multiple order surface differential quantities. In: 18th IEEE International Conference on Image Processing (2011)

Download references

Author information

Authors and Affiliations

  1. Department of Management and Production Engineering, Politecnico di Torino, Corso Duca degli Abruzzi 24, 10129, Turin, Italy

    Nicole Dagnes, Federica Marcolin, Francesca Nonis, Stefano Tornincasa & Enrico Vezzetti

Authors
  1. Nicole Dagnes
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Federica Marcolin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Francesca Nonis
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Stefano Tornincasa
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Enrico Vezzetti
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Federica Marcolin.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Dagnes, N., Marcolin, F., Nonis, F. et al. 3D geometry-based face recognition in presence of eye and mouth occlusions. Int J Interact Des Manuf 13, 1617–1635 (2019). https://doi.org/10.1007/s12008-019-00582-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12008-019-00582-7

Keywords

Search

Navigation