Abstract
Convolutional Neural Networks are a popular approach for image classification problem. This article presents an overview of open-source facial expression datasets and performance comparison of CNN models. Evaluated model, trained to detect seven basic emotions, on the combined set of datasets offers 86,7% of accuracy on a validation set and 97,2% on a training set. In this work a system of automated appropriate response to human emotion expression and set of layers that ensure high performance are proposed. The system combines real-time CNN with robotic head OhBot. The information about current emotional state of a person based on its facial expression is the input signal for the subsystem controlling the robotic head, whose task is to react appropriate to the situation.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Eryka, P., et al.: Application of tiny-ML methods for face recognition in social robotics using OhBot robots, pp. 146–151 (2022). https://doi.org/10.1109/MMAR55195.2022.9874278
Darwin, C.: The Expression of the Emotions in Man and Animals, 3rd edn. Fontana Press, London (1999/1872)
Ekman, P., Sorenson, E.R., Friesen, W.V.: Pan-cultural elements in facial displays of emotion. Science 164, 86–88 (1969)
Fong, T., Nourbakhsh, I., Dautenhahn, K.: A survey of socially interactive robots. Robot. Auton. Syst. 42, 143–166 (2003). https://doi.org/10.1016/S0921-8890(02)00372-X
Kirby, R., Forlizzi, J., Simmons, R.: Affective social robots. Robot. Auton. Syst. 58(3), 322–332 (2010)
Lugaresi, C., et al.: MediaPipe: A Framework for Building Perception Pipelines (2019). https://google.github.io/mediapipe/ (term. wiz. 03 Jan 2023)
Fasola, J., Matarić, M.J.: A socially assistive robot exercise coach for the elderly. 2(2), 3–32 (2013). https://doi.org/10.5898/JHRI.2.2.Fasola
IMotions Facial Expression Analysis. https://imotions.com/facial-expressions. Accessed 12 Dec 2018
Abramson, L., Marom, I., Petranker, R., Aviezer, H.: Is fear in your head? A comparison of instructed and real-life expressions of emotion in the face and body. Emotion 17, 557–565 (2017)
Magdin, M., Benko, L., Koprda, Š.: A case study of facial emotion classification using affdex. Sensors 19, 2140 (2019). https://doi.org/10.3390/s19092140
Yaermaimaiti, Y., Kari, T., Zhuang, G.: Research on facial expression recognition based on an improved fusion algorithm. Nonlinear Eng. 11(1), 112–122 (2022). https://doi.org/10.1515/nleng-2022-0015
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: a large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255. IEEE (2009)
Simonyan, K., Zisserman, A.: Very Deep Convolutional Networks for Large-Scale Image Recognition (2014). https://doi.org/10.48550/ARXIV.1409.1556
He, K., Zhang, X., Ren, S., Sun, J.: Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification (2015). https://doi.org/10.48550/ARXIV.1502.01852
Saste, S.T., Jagdale, S.M.: Emotion recognition from speech using MFCC and DWT for security system. In: 2017 International Conference of Electronics, Communication and Aerospace Technology (ICECA), vol. 1, pp. 701–704 (2017)
Sun, B., Wei, Q., Li, L., Xu, Q., He, J., Yu, L.: LSTM for dynamic emotion and group emotion recognition in the wild. In: Proceedings of the 18th ACM International Conference on Multimodal Interaction, pp. 451–457. Association for Computing Machinery, New York, NY, USA (2016). ISBN 9781450345569
Kim, J.-C., Kim, M.-H., Suh, H.-E., Naseem, M., Lee, C.-S.: Hybrid approach for facial expression recognition using convolutional neural networks and SVM. Appl. Sci. 12, 5493 (2022). https://doi.org/10.3390/app12115493
Elissa, K.: Toward socially assistive robotics for augmenting interventions for children with autism spectrum disorders. In: Khatib, O., Kumar, V., Pappas, G.J. (eds.) Experimental Robotics. Springer Tracts in Advanced Robotics, vol. 54, pp. 201–210. Springer, Heidelberg (2009). https://doi.org/10.1007/978-3-642-00196-3_24
Acknowledgements
The work was supported by Mentoring Program realised by the Silesian University of Technology (SUT) (Program Mentorski - “Rozwin skrzydla”) and paid from the reserve of the Vice-Rector for Student Affairs and Education: MPK: 60/001 GŹF: SUBD. This work was supported by Upper Silesian Centre for Computational Science and Engineering (GeCONiI) through The National Centre for Research and Development (NCBiR) under Grant POIG.02.03.01-24-099/13. This work was supported by Rector’s funds within 4th competition for financing projects of student research clubs under the Initiative of Excellence - Research University in the year 2023.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Bartosiak, N., Gałuszka, A., Wojnar, M. (2023). Implementation of a Neural Network for the Recognition of Emotional States by Social Robots, Using ‘OhBot’. In: Rojas, I., Joya, G., Catala, A. (eds) Advances in Computational Intelligence. IWANN 2023. Lecture Notes in Computer Science, vol 14135. Springer, Cham. https://doi.org/10.1007/978-3-031-43078-7_15
Download citation
DOI: https://doi.org/10.1007/978-3-031-43078-7_15
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-43077-0
Online ISBN: 978-3-031-43078-7
eBook Packages: Computer ScienceComputer Science (R0)