Skip to main content

Advertisement

SpringerLink
Log in

Recognition of surgically altered face images: an empirical analysis on recent advances

  • Published:
Artificial Intelligence Review Aims and scope Submit manuscript

Abstract

Biometric recognition plays a vital role in our daily lives. Face recognition is a subset of biometric recognition. Face verification and identification processes are prone to plastic surgery challenges which are commonly used nowadays to alter facial features for good looking demonstration. With increasing trend in technology and intellect robust biometric recognition systems are developed for human recognition after plastic surgery. However, these systems have some limitations because recognition after plastic surgery is affected by lightning, aging, pose, expressions, disguise and occlusion effects. In this survey, we aim to highlight the mitigating effects of cutting edge plastic surgical operations. These procedures lead to medical identity thefts, which is a serious offense for human community as an individual’s identity is forged. Thus, this makes one’s safety a critical issue and human recognition after plastic surgery a crucial challenge. Since the existing methods for human recognition after plastic surgical operations are not promising, in the current scenario plastic surgical operations secure above facial recognition. A number of existing biometric recognition algorithms for face images have been opted such as principal component analysis, Fisher/linear discriminant analysis, local feature analysis, local/circular binary patterns, speeded up robust features, granular system, correlation based approach, evolutionary granular/genetic approach, grouping recognition by parts and sparse demonstration approach, geometrical face recognition after plastic surgery, feature/texture based fusion scheme and deep convolutional neural networks (DCNN). The validation metrics used for the evaluation of recognition techniques are expected error rate, recognition rate, half total error rate and F-score. All algorithms are tested on an open plastic surgery facial dataset containing 1800 before and after surgery image samples pertaining to 900 humans. For a particular human being, two front facing image samples with appropriate luminance and unbiased gesture are taken: the former is taken pre cosmetic procedure and the latter is taken post cosmetic procedure. It has been deduced that feature and texture based fusion approach gives best results till date. It is predicted that DCNN has full potential of giving consistent results on surgical databases as it is already validated on non surgical databases. The need of a novel human identification system which is steady to the anomalies posed by plastic surgical operations is highlighted in this survey.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23
Fig. 24
Fig. 25
Fig. 26
Fig. 27

Similar content being viewed by others

References

  • Aggarwal G, Biswas S, Flynn PJ et al (2012) A sparse representation approach to face matching across plastic surgery. In: IEEE workshop on applications of computer vision (WACV 2012), pp 113–119

  • Ailon T, Sure DR, Smith JS, Shaffrey CI (2016) Surgical considerations for major deformity correction spine surgery. Best Pract Res Clin Anaesthesiol 30(1):3–11

    Article  Google Scholar 

  • Ali ASO, Sagayan V, Malik A, Aziz A (2016) Proposed face recognition system after plastic surgery. IET Comput Vis 10(5):342–348

    Article  Google Scholar 

  • Ali M, Dat LQ, Son LH, Smarandache F (2018a) Interval complex neutrosophic set: formulation and applications in decision-making. Int J Fuzzy Syst 20(3):986–999

    Article  Google Scholar 

  • Ali M, Son LH, Thanh ND, Van Minh N (2018b) A neutrosophic recommender system for medical diagnosis based on algebraic neutrosophic measures. Appl Soft Comput. https://doi.org/10.1016/j.asoc.2017.10.012

    Google Scholar 

  • Arslan B, Yorulmaz E, Akca B, Sagiroglu S (2016) Security perspective of biometric recognition and machine learning techniques. In: 2016 15th IEEE international conference on machine learning and applications (ICMLA), pp 492–497

  • Bhatt HS, Bharadwaj S, Singh R, Vatsa M (2013) Recognizing surgically altered face images using multiobjective evolutionary algorithm. IEEE Trans Inf Forens Secur 8(1):89–100

    Article  Google Scholar 

  • Chappell AG, Lalikos JF (2016) An old idea revisited: reflections on the role of aesthetic surgery in behavioral and social change. Plast Reconstr Surg 138(3):568e–569e

    Article  Google Scholar 

  • Chawla KS, Rutkow L, Garber K, Kushner AL, Stewart BT (2017) Beyond a moral obligation: a legal framework for emergency and essential surgical care and anesthesia. World J Surg 41(5):1208–1217

    Article  Google Scholar 

  • Chen Y, Yang J, Wang C, Liu N (2016) Multimodal biometrics recognition based on local fusion visual features and variational Bayesian extreme learning machine. Expert Syst Appl 64:93–103

    Article  Google Scholar 

  • Dey A, Broumi S, Son LH, Bakali A, Talea M, Smarandache F (2018) A new algorithm for finding minimum spanning trees with undirected neutrosophic graphs. Granul Comput. https://doi.org/10.1007/s41066-018-0084-7

    Google Scholar 

  • De Marsico M, Nappi M, Riccio D, Wechsler H (2011, June) Robust face recognition after plastic surgery using local region analysis. In: International conference image analysis and recognition (pp. 191–200). Springer, Berlin

  • Del Rio JS, Moctezuma D, Conde C, de Diego IM, Cabello E (2016) Automated border control e-gates and facial recognition systems. Comput Secur 62:49–72

    Article  Google Scholar 

  • Denman S, Halstead M, Fookes C, Sridharan S (2017) Locating people in surveillance video using soft biometric traits. In: Tistarelli M, Champod C (eds) Handbook of biometrics for forensic science. Springer, Cham, pp 267–288

    Chapter  Google Scholar 

  • Dong Y, Li Y, Sun T (2014) Happy faces considered trustworthy irrespective of perceiver’s mood: challenges to the mood congruency effect. Comput Secur 47:85–93

    Article  Google Scholar 

  • El Said SA, Abol Atta HM (2014) Geometrical face recognition after plastic surgery. Int J Comput Appl Technol 49(3/4):352–364

    Article  Google Scholar 

  • Face Databases (2017) AT&T, ORL and YALE databases. http://www.face-rec.org/databases/. Accessed 20 Dec 2017

  • Geng X, Zhou Z-H, Smith-Miles K (2007) Automatic age estimation based on facial aging patterns. IEEE Trans Pattern Anal Mach Intell 29(12):2234–2240

    Article  Google Scholar 

  • Giap CN, Son LH, Chiclana F (2018) Dynamic structural neural network. J Intell Fuzzy Syst 4(2479–2490):2

    Google Scholar 

  • Golshani S, Mani A, Toubaei S, Farnia V, Sepehry AA, Alikhani M (2016) Personality and psychological aspects of cosmetic surgery. Aesthet Plast Surg 40(1):38–47

    Article  Google Scholar 

  • Grm K, Štruc V, Artiges A, Caron M, Ekenel H (2017) Strengths and weaknesses of deep learning models for face recognition against image degradations. IET Biomet Spec Issue Face Recognit Spoof Attacks 7(1):81–89

    Google Scholar 

  • Haghighat M, Abdel-Mottaleb M, Alhalabi W (2016) Fully automatic face normalization and single sample face recognition in unconstrained environments. Expert Syst Appl 47:23–34

    Article  Google Scholar 

  • Hemanth DJ, Anitha J, Son LH (2018) Brain signal based human emotion analysis by circular back propagation and Deep Kohonen Neural Networks. Comput Electr Eng 68:170–180

    Article  Google Scholar 

  • Koch W, Rettig EM, Sun DQ (2017) Head and neck essentials in global surgery. In: Park A, Price R (eds) Surgery global. Springer, Cham, pp 443–474

    Chapter  Google Scholar 

  • Lahasan B, Lutfi SL, San-Segundo R (2017) A survey on techniques to handle face recognition challenges: occlusion, single sample per subject and expression. Artif Intell Rev. https://doi.org/10.1007/s10462-017-9578-y

    Google Scholar 

  • Liu L, Fieguth P, Zhao G, Pietikäinen M, Hu D (2016) Extended local binary patterns for face recognition. Inf Sci 358:56–72

    Article  Google Scholar 

  • Mesa J, Lalonde D, Vasconez LO (2017) Local and regional anesthesia in plastic surgery: safety considerations and management of adverse events. In: Finucane BT, Tsui BCH (eds) Complications of regional anesthesia. Springer, Cham, pp 399–409

    Chapter  Google Scholar 

  • Micheals RJ, Boult TE (2001) Efficient evaluation of classification and recognition systems. In: Proceedings of the 2001 IEEE computer society conference on computer vision and pattern recognition, vol 1, pp 1–50

  • Mohammadi S, Ahmadi A, Salem MM, Safdarian M, Ilkhani S (2015) A comparison between two methods of face-lift surgery in nine cadavers: SMAS (superficial musculo-aponeurotic system) versus MACS (minimal access cranial suspension). Aesthet Plast Surg 39(5):680–685

    Article  Google Scholar 

  • Morzycki A, Williams J (2016) The Canadian contribution to the global plastic surgery literature: a 10-year bibliometric analysis. Plast Reconstruct Surg Glob Open 4(9S):229–230

    Article  Google Scholar 

  • Nappi M, Ricciardi S, Tistarelli M (2016) Deceiving faces: when plastic surgery challenges face recognition. Image Vis Comput 54:71–82

    Article  Google Scholar 

  • Neves J, Narducci F, Barra S, Proença H (2016) Biometric recognition in surveillance scenarios: a survey. Artif Intell Rev 46(4):515–541

    Article  Google Scholar 

  • Ngan TT, Tuan TM, Son LH, Minh NH, Dey N (2016) Decision making based on fuzzy aggregation operators for medical diagnosis from dental X-ray images. J Med Syst 40(12):280

    Article  Google Scholar 

  • Ngan RT, Son LH, Cuong BC, Ali M (2018) H-max distance measure of intuitionistic fuzzy sets in decision making. Appl Soft Comput 69:393–425

    Article  Google Scholar 

  • Nguyen GN, Ashour AS, Dey N (2017) A survey of the state-of-the-arts on neutrosophic sets in biomedical diagnoses. Int J Mach Learn Cybern. https://doi.org/10.1007/s13042-017-0691-7

    Google Scholar 

  • Oravec M (2014) Feature extraction and classification by machine learning methods for biometric recognition of face and iris. In: 2014 56th IEEE international symposium on ELMAR (ELMAR), pp 1–4

  • Park U, Tong Y, Jain AK (2010) Age-invariant face recognition. IEEE Trans Pattern Anal Mach Intell 32(5):947–954

    Article  Google Scholar 

  • Plastic surgery face database (2017) http://iiitd.edu.in/iab/Image_Analysis_and_Biometrics_Group/Resources.html/, http://iab-rubric.org/resources.html/

  • Ren CX, Lei Z, Dai DQ, Li SZ (2016) Enhanced local gradient order features and discriminant analysis for face recognition. IEEE Trans Cybern 46(11):2656–2669

    Article  Google Scholar 

  • Ricanek K (2013) The next biometric challenge: medical alterations. IEEE Computer Society, pp 94–96

  • Savchenko AV (2016) Fast multi-class recognition of piecewise regular objects based on sequential three-way decisions and granular computing. Knowl Based Syst 91:252–262

    Article  Google Scholar 

  • Schryen G, Wagner G, Schlegel A (2016) Development of two novel face-recognition CAPTCHAs: a security and usability study. Comput Secur 60:95–116

    Article  Google Scholar 

  • Seo HJ, Milanfar P (2011) Face verification using the lark representation. IEEE Trans Inf Foren Sec 6(4):1275–1286

    Article  Google Scholar 

  • Shi Y, Ren X, Yang S, Gong P (2016) A generalized kernel fisher discriminant framework used for feature extraction and face recognition. In: 2016 12th IEEE international conference on natural computation, fuzzy systems and knowledge discovery (ICNC-FSKD), pp 1487–1491

  • Singh R, Vatsa M, Bhatt HS, Bharadwaj S, Noore A, Nooreyezdan SS (2010) Plastic surgery: a new dimension to face recognition. IEEE Trans Inf Forensics Secur 5(3):441–448

    Article  Google Scholar 

  • Son LH, Fujita H (2018) Neural-fuzzy with representative sets for prediction of student performance. Appl Intell. https://doi.org/10.1007/s10489-018-1262-7

    Google Scholar 

  • Son LH, Phong PH (2016) On the performance evaluation of intuitionistic vector similarity measures for medical diagnosis. J Intell Fuzzy Syst 31(3):1597–1608

    Article  MATH  Google Scholar 

  • Son LH, Tuan TM (2017) Dental segmentation from X-ray images using semi-supervised fuzzy clustering with spatial constraints. Eng Appl Artif Intell 59:186–195

    Article  Google Scholar 

  • Son LH, Chiclana F, Kumar R, Mittal M, Khari M, Chatterjee JM, Baik SW (2018) ARM–AMO: an efficient association rule mining algorithm based on animal migration optimization. Knowl-Based Syst 154:68–80

    Article  Google Scholar 

  • Su Y, Shan S, Chen X, Gao W (2009) Hierarchical ensemble of global and local classifiers for face recognition. IEEE Trans Image Process 18(8):1885–1896

    Article  MathSciNet  MATH  Google Scholar 

  • Talisman R (2014) Patient courage leads us to wonder: should we perform face-lifts on patients taking coumadin? Aesthet Plast Surg 38(2):442

    Article  Google Scholar 

  • Thanh ND, Ali M, Son LH (2017) A novel clustering algorithm in a neutrosophic recommender system for medical diagnosis. Cognit Comput 9(4):526–544

    Article  Google Scholar 

  • Tuong L, Son LH, Vo MT, Lee MY, Baik SW (2018) Cluster-based boosting algorithm for bankruptcy prediction. Symmetry Basel 10:250–262

    Article  Google Scholar 

  • Wang Z, Ruan Q, An G (2016) Facial expression recognition using sparse local Fisher discriminant analysis. Neurocomputing 174:756–766

    Article  Google Scholar 

  • Zahradnikova B, Duchovicova S, Schreiber P (2018) Facial composite systems. Artif Intell Rev 49(1):131–152

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. USICT, GGSIPU, Dwarka, Delhi, India

    Tanupreet Sabharwal

  2. Northern India Engineering College, New Delhi, Delhi, India

    Tanupreet Sabharwal

  3. Department of Electronics and Communication Engineering, Ambedkar Institute of Advanced Communication Technologies and Research, New Delhi, Delhi, India

    Rashmi Gupta

  4. VNU Information Technology Institute, Vietnam National University, Hanoi, Vietnam

    Le Hoang Son

  5. Computer Science and Engineering Department, LNCT College, Bhopal, MP, India

    Raghvendra Kumar

  6. School of Computer Engineering, KIIT University, Odisha, India

    Sudan Jha

Authors
  1. Tanupreet Sabharwal
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Rashmi Gupta
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Le Hoang Son
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Raghvendra Kumar
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Sudan Jha
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Le Hoang Son.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sabharwal, T., Gupta, R., Son, L.H. et al. Recognition of surgically altered face images: an empirical analysis on recent advances. Artif Intell Rev 52, 1009–1040 (2019). https://doi.org/10.1007/s10462-018-9660-0

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10462-018-9660-0

Keywords

Search

Navigation