Abstract
Spontaneous facial expressions differ from posed ones in appearance, timing and accompanying head movements. Still images cannot provide timing or head movement information directly. However, indirectly the distances between key points on a face extracted from a still image using active shape models can capture some movement and pose changes. This information is superposed on information about non-rigid facial movement that is also part of the expression. Does geometric information improve the discrimination between spontaneous and posed facial expressions arising from discrete emotions? We investigate the performance of a machine vision system for discrimination between posed and spontaneous versions of six basic emotions that uses SIFT appearance based features and FAP geometric features. Experimental results on the NVIE database demonstrate that fusion of geometric information leads only to marginal improvement over appearance features. Using fusion features, surprise is the easiest emotion (83.4% accuracy) to be distinguished, while disgust is the most difficult (76.1%). Our results find different important facial regions between discriminating posed versus spontaneous version of one emotion and classifying the same emotion versus other emotions. The distribution of the selected SIFT features shows that mouth is more important for sadness, while nose is more important for surprise, however, both the nose and mouth are important for disgust, fear, and happiness. Eyebrows, eyes, nose and mouth are important for anger.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Cohn, J., Schmidt, K.: The Timing of Facial Motion in Posed and Spontaneous Smiles. International Journal of Wavelets, Multiresolution and Information Processing 2, 1–12 (2004)
Hamdi, D., Roberto, V., Albert Ali, S., Theo, G.: Eyes Do Not Lie: Spontaneous versus Posed Smiles. In: Proceedings of the International Conference on Multimedia, pp. 703–706. ACM, Firenze (2010)
Michel, F.V., Hatice, G., Maja, P.: How to Distinguish Posed from Spontaneous Smiles Using Geometric Features. In: Proceedings of the 9th International Conference on Multimodal Interfaces, pp. 38–45. ACM, Nagoya (2007)
Michel, F.V., Maja, P., Zara, A., Jeffrey, F.C.: Spontaneous vs. Posed Facial Behavior: Automatic Analysis of Brow Actions. In: Proceedings of the 8th International Conference on Multimodal Interfaces, pp. 162–170. ACM, Banff (2006)
Littlewort, G.C., Bartlett, M.S., Lee, K.: Automatic Coding of Facial Expressions Displayed During Posed and Genuine Pain. Image and Vision Computing 27, 1797–1803 (2009)
Bartlett, M., Littlewort, G., Vural, E., Lee, K., Cetin, M., Ercil, A., Movellan, J.: Data Mining Spontaneous Facial Behavior with Automatic Expression Coding. In: Esposito, A., Bourbakis, N.G., Avouris, N., Hatzilygeroudis, I. (eds.) HH and HM Interaction. LNCS (LNAI), vol. 5042, pp. 1–20. Springer, Heidelberg (2008)
Mingli, S., Dacheng, T., Zicheng, L., Xuelong, L., Mengchu, Z.: Image Ratio Features for Facial Expression Recognition Application. IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics 40, 779–788 (2010)
Yuxiao, H., Zhihong, Z., Lijun, Y., Xiaozhou, W., Xi, Z., Huang, T.S.: Multi-view Facial Expression Recognition. In: 8th IEEE International Conference on Automatic Face & Gesture Recognition, FG 2008, pp. 1–6 (2008)
Hao, T., Huang, T.S.: 3D Facial Expression Recognition Based on Automatically Selected Features. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, CVPRW 2008, pp. 1–8 (2008)
Cootes, T.F., Taylor, C.J., Cooper, D.H., Graham, J.: Active Shape Models-Their Training and Application. Comput. Vis. Image Underst. 61, 38–59 (1995)
Lowe, D.G.: Distinctive Image Features from Scale-Invariant Keypoints. International Journal of Computer Vision 60, 91–110 (2004)
Berretti, S., Bimbo, A.D., Pala, P., Amor, B.B., Daoudi, M.: A Set of Selected SIFT Features for 3D Facial Expression Recognition. In: 20th International Conference on Pattern Recognition, ICPR 2010, pp. 4125–4128 (2010)
Pandzic, I.S., Forchheimer, R.: MPEG-4 facial animation: the standard, implementation and applications. Wiley (2002)
Hanchuan, P., Fuhui, L., Ding, C.: Feature Selection Based on Mutual Information Criteria of Max-dependency, Max-relevance, and Min-redundancy. IEEE Transactions on Pattern Analysis and Machine Intelligence 27, 1226–1238 (2005)
Lajevardi, S., Hussain, Z.: Automatic Facial Expression Recognition: Feature Extraction and Selection. Signal, Image and Video Processing, 1–11 (2010)
Shangfei, W., Zhilei, L., Siliang, L., Yanpeng, L., Guobing, W., Peng, P., Fei, C., Xufa, W.: A Natural Visible and Infrared Facial Expression Database for Expression Recognition and Emotion Inference. IEEE Transactions on Multimedia 12, 682–691 (2010)
Schmidt, K., Ambadar, Z., Cohn, J., Reed, L.: Movement Differences between Deliberate and Spontaneous Facial Expressions: Zygomaticus Major Action in Smiling. Journal of Nonverbal Behavior 30, 37–52 (2006)
Schmidt, K., Bhattacharya, S., Denlinger, R.: Comparison of Deliberate and Spontaneous Facial Movement in Smiles and Eyebrow Raises. Journal of Nonverbal Behavior 33, 35–45 (2009)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Zhang, L., Tjondronegoro, D., Chandran, V. (2011). Geometry vs. Appearance for Discriminating between Posed and Spontaneous Emotions. In: Lu, BL., Zhang, L., Kwok, J. (eds) Neural Information Processing. ICONIP 2011. Lecture Notes in Computer Science, vol 7064. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-24965-5_49
Download citation
DOI: https://doi.org/10.1007/978-3-642-24965-5_49
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-24964-8
Online ISBN: 978-3-642-24965-5
eBook Packages: Computer ScienceComputer Science (R0)