Abstract
An important challenge in building automatic affective state recognition systems is establishing the ground truth. When the ground-truth is not available, observers are often used to label training and testing sets. Unfortunately, inter-rater reliability between observers tends to vary from fair to moderate when dealing with naturalistic expressions. Nevertheless, the most common approach used is to label each expression with the most frequent label assigned by the observers to that expression. In this paper, we propose a general pattern recognition framework that takes into account the variability between observers for automatic affect recognition. This leads to what we term a multi-score learning problem in which a single expression is associated with multiple values representing the scores of each available emotion label. We also propose several performance measurements and pattern recognition methods for this framework, and report the experimental results obtained when testing and comparing these methods on two affective posture datasets.
The original version of this chapter was revised. An erratum forthis chapter can be found at: http://dx.doi.org/10.1007/978-3-642-24600-5_66
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Argyriou, A., Evgeniou, T., Pontil, M.: Convex multi-task feature learning. Machine Learning 73, 243–272 (2008)
Camurri, A., Lagerlof, I., Volpe, G.: Recognizing emotion from dance movement: Comparison of spectator recognition and automated techniques. International Journal of Human-Computer Studies 59(1-2), 213–225 (2003)
Castellano, G., Karpouzis, K., Peters, C., Martin, J.-C. (eds.): Special Issue on Real-Time Affect Analysis and Interpretation: Closing the Affective Loop in Virtual Agents and Robots. Journal on Multimodal User Interfaces 3(1), 1–3 (2010)
Chittaranjan, G., Aran, O., Gatica-Perez, D.: Exploiting observers judgements for nonverbal group interaction. In: Proceedings of IEEE International Conference on Automatic Face and Gesture Recognition (FG 2011), Santa Barbara, CA, USA (March 2011)
Jong, S.: Simpls: An alternative approach to partial least squares regression. Chemometrics and Intelligent Laboratory Systems 18(3), 251–263 (1993)
Drucker, H., Burges, C., Kaufman, L., Smola, A., Vapnik, V.: Support vector regression machines. In: Advances in Neural Information Processing Systems, vol. 9, pp. 155–161 (1997)
Fürnkranz, J., Hüllermeier, E.: Pairwise preference learning and ranking. In: Lavrač, N., Gamberger, D., Todorovski, L., Blockeel, H. (eds.) ECML 2003. LNCS (LNAI), vol. 2837, pp. 145–156. Springer, Heidelberg (2003)
Kleinsmith, A.: Grounding Affect Recognition on a Low-Level Description of Body Posture. Ph.D. Thesis. University College London (2010)
Kleinsmith, A., Bianchi-Berthouze, N., Steed, A.: Automatic recognition of non-acted affective postures. IEEE Transactions on Systems, Man and Cybernetics, Part B 99, 1–12 (2011)
Kleinsmith, A., de Silva, P.R., Bianchi-Berthouze, N.: Cross-cultural differences in recognizing affect from body posture. Interacting with Computers 18(6), 1371–1389 (2006)
Mandryk, R.L., Inkpen, K.M., Calvert, T.W.: Using psychophysiological techniques to measure user experience with entertainment technologies. Behaviour & IT 25(2), 141–158 (2006)
Mardia, K.V., Kent, J.T., Bibby, J.M.: Multivariate Analysis. Academic Press, London (1980)
Moody, J., Darken, C.J.: Fast learning in networks of locally-tuned processing units. Neural Comput. 1(2), 281–294 (1989)
Nicolaou, M.A., Gunes, H., Pantic, M.: Output-associative rvm regression for dimensional and continuous emotion prediction. In: Proceedings of IEEE International Conference on Automatic Face and Gesture Recognition (FG 2011), Santa Barbara, CA, USA (March 2011)
Pantic, M., Rothkrantz, L.J.M.: Automatic analysis of facial expressions: The state of the art. IEEE Transactions on Pattern Analysis and Machine Intelligence 22, 1424–1445 (2000)
Paterson, H.M., Pollick, F.E., Sanford, A.J.: The role of velocity in affect discrimination. In: Proceedings of the 23rd Annual Conference of the Cognitive Science Society, pp. 756–761. Lawrence Erlbaum Associates, Mahwah (2001)
Picard, R.W.: Affective Computing. The MIT Press, Cambridge (1997)
Rosipal, R., Krämer, N.: Overview and recent advances in partial least squares. In: Saunders, C., Grobelnik, M., Gunn, S., Shawe-Taylor, J. (eds.) SLSFS 2005. LNCS, vol. 3940, pp. 34–51. Springer, Heidelberg (2006)
Salton, G.: Developments in automatic text retrieval. Science 253(5023), 974–980 (1991)
De Silva, P.R., Bianchi-Berthouze, N.: Modeling human affective postures:an information theoretic characterization of posture features. Computer Animation and Virtual Worlds 15(3-4), 269–276 (2004)
Specht, D.F.: A general regression neural network. IEEE Transactions on Neural Networks 2(6), 568–576 (1991)
Wagner, J., Andre, E., Lingenfelser, F., Kim, J., Vogt, T.: Exploring Fusion Methods for Multimodal Emotion Recognition with Missing Data. IEEE Transactions on Affective Computing 99, 1949–3045 (2011)
Zeng, Z., Pantic, M., Roisman, G.I., Huang, T.S.: A survey of affect recognition methods: Audio, visual, and spontaneous expressions. IEEE Trans. Pattern Anal. Mach. Intell. 31(1), 39–58 (2009)
Zhang, M., Zhou, Z.: ML-KNN: A lazy learning approach to multi-label learning. Pattern Recognition 40(7), 2038–2048 (2007)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Meng, H., Kleinsmith, A., Bianchi-Berthouze, N. (2011). Multi-score Learning for Affect Recognition: The Case of Body Postures. In: D’Mello, S., Graesser, A., Schuller, B., Martin, JC. (eds) Affective Computing and Intelligent Interaction. ACII 2011. Lecture Notes in Computer Science, vol 6974. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-24600-5_26
Download citation
DOI: https://doi.org/10.1007/978-3-642-24600-5_26
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-24599-2
Online ISBN: 978-3-642-24600-5
eBook Packages: Computer ScienceComputer Science (R0)