Abstract
Estimating the attractiveness of faces in images and videos is a relatively new problem in computer vision. So far, supervised learning paradigms with deep or shallow models have been the mainstream. In this work, we explore a semi-supervised paradigm that is very useful and practical when there are not so many scored images. The proposed semi-supervised method fuses multiple graphs to find a unified flexible manifold embedding model. This model provides the attractiveness score of the unlabeled face images and a linear mapping that maps the feature space to the score space. The proposed method merges the graphs of geometric features and deep features to estimate a unified embedding. It also improves the discriminative power of the graph-based score propagation method by creating an additional similarity graph from the predicted scores. The experiments were performed on the public SCUTFBP-5500 facial beauty dataset. They show that the proposed approach performs well compared to other state-of-the-art methods.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Data availability statement
The dataset SCUT-FBP5500 analyzed during the current study is available in the github repository, https://github.com/HCIILAB/SCUT-FBP5500-Database-Release
References
An L, Chen X, Yang S (2017) Multi-graph feature level fusion for person re-identification. Neurocomput Multimodal Media Data Underst Anal 259:39–45
An L, Chen X, Yang S (2017) Multi-graph feature level fusion for person re-identification. Neurocomput Multimodal Media Data Underst Anal 259:39–45
Cao Q, Ying Y, Li P (2013) Similarity metric learning for face recognition. In Proceedings of the IEEE International Conference on Computer Vision (ICCV),
Dornaika F, Dahbi R, Bosaghzadeh A, Ruichek Y (2017) Efficient dynamic graph construction for inductive semi-supervised learning. Neural Netw 94:192–203
Dornaika F, Elorza A, Wang K, Arganda-Carreras I (2019) Nonlinear, flexible, semisupervised learning scheme for face beauty scoring. J Electron Imag 28(1):07
Eisenthal Y, Dror G, Ruppin E (2006) Facial attractiveness: beauty and the machine. Neural Comput 18(1):119–142
Eppstein D, Paterson M, Yao F (1997) On nearest-neighbor graphs. Comput Geom 17(4):263–282
Gan J, Li L, Zhai Y, Liu Y (2014) Deep self-taught learning for facial beauty prediction. Neurocomputing 144:295–303
Gray D, Yu K, Xu W, Gong Y (2010) Predicting facial beauty without landmarks. Comput Vis ECCV 2010:434–447
Gunes H, Piccardi M (2006) Assessing facial beauty through proportion analysis by image processing and supervised learning. Int J Human-Comput Stud 64(12):1184–1199
He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 770–778,
Hossam M, Afify AA, Rady M, Nabil v, Moussa R, Yousri Darweesh MS (2021) A comparative study of different face shape classification techniques. In 2021 International Conference on Electronic Engineering (ICEEM), pages 1–6,
Karasuyama M, Mamitsuka H (2013) Multiple graph label propagation by sparse integration. IEEE Trans Neural Netw Learn Syst 24(12):1999–2012
Langlois JH, Roggman LA (1990) Attractive faces are only average. Psychol Sci 1(2):115–121
Lin G, Liao K, Sun B, Chen Y, Zhao F (2017) Dynamic graph fusion label propagation for semi-supervised multi-modality classification. Pattern Recogn 68:14–23
Liu X, Li T, Peng H, Ouyang IC, T. Kim, and R. Wang (2019) Understanding beauty via deep facial features. In 2019 IEEE/CVF Conference on Computer Vision and Pattern Rec(zognition Workshops (CVPRW), 246–256,
Namjoy A, Bosaghzadeh A (2020) A sample dependent decision fusion algorithm for graph-based semi-supervised learning. Int J Eng 33(5):1010–1019
Nie F, Xu D, Tsang IW, Zhang C (2010) Flexible manifold embedding: a framework for semi-supervised and unsupervised dimension reduction. IEEE Trans Imag Process 19(7):1921–1932
Nie F, Xu D, Tsang IW-H, Zhang C (2010) Flexible manifold embedding: a framework for semi-supervised and unsupervised dimension reduction. IEEE Trans Imag Process 19(7):1921–1932
Parkhi OM, Vedaldi A, Zisserman A et al (2015) Deep face recognition. In BMVC, volume 1, page 6,
Saeed JN, Abdulazeez AM (2021) Facial beauty prediction and analysis based on deep convolutional neural network: a review. J Soft Comput Data Min 2(1):4
B. Schölkopf, J. Platt, and T. Hofmann. A Humanlike Predictor of Facial Attractiveness, pages 649–656. MIT Press, 2007
Wang B, Tsotsos J (2016) Dynamic label propagation for semi-supervised multi-class multi-label classification. Pattern Recogn 52:75–84
Wang S, Shao M, and Fu Y (2014) Attractive or not?: Beauty prediction with attractiveness-aware encoders and robust late fusion. In Proceedings of the 22nd ACM international conference on Multimedia, pages 805–808. ACM,
Zhang D, Zhao Q, Chen F (2011) Quantitative analysis of human facial beauty using geometric features. Pattern Recogn 44(4):940–950
Zhang D, Zhao Q, Chen F (2011) Quantitative analysis of human facial beauty using geometric features. Pattern Recogn 44:940–950
Zhang L, Zhang D (2016) Metricfusion: generalized metric swarm learning for similarity measure. Inf Fusion 30(80–90):07
Zhang Y, Zhang H, Nasrabadi NM, Huang TS (2013) Multi-metric learning for multi-sensor fusion based classification. Inf Fusion 14(4):431–440
Zhou D, Bousquet O, Lal T, Weston J, and Schölkopf B. Learning with local and global consistency. In Thrun S, Saul L, and Schölkopf B, editors, Advances in Neural Information Processing Systems, volume 16. MIT, 2003
Zhu R, Dornaika F, Ruichek Y (2019) Learning a discriminant graph-based embedding with feature selection for image categorization. Neural Netw 111:35–46
Ziraki N, Dornaika F, Bosaghzadeh A (2022) Multiple-view flexible semi-supervised classification through consistent graph construction and label propagation. Neural Netw 146:174–180
Funding
The authors declare that no funds, grants, or other support were received during the preparation of this manuscript.
Author information
Authors and Affiliations
Contributions
All authors contributed to the study conception and design. Material preparation, data collection and analysis were performed by Fadi Dornaika.
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Ethical approval
This article complies with ethical standards. This article does not include studies with human participants or animals conducted by any of the authors.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Dornaika, F. Multi-similarity semi-supervised manifold embedding for facial attractiveness scoring. Soft Comput 27, 5099–5108 (2023). https://doi.org/10.1007/s00500-023-07963-x
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00500-023-07963-x