Abstract
Facial emotion recognition plays an important role in day-to-day activities. To address this, we propose a novel encoder/decoder network namely EmotionCaps, which models the facial images using matrix capsules, where hierarchical pose relationships between facial parts are built into internal representations. An optimal number of capsules and their dimension is chosen, as these hyper-parameters in the network play an important role to capture the complex facial pose relationship. Further, the batch normalization layer is introduced to expedite the convergence. To show the effectiveness of our network, EmotionCaps is evaluated for seven basic emotions in a wide range of head orientations. Additionally, our method is able to analyze facial images even in the presence of noise and blur quite accurately.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Abien Fred, M.A.: Deep learning using rectified linear units (ReLU). Neural Evol. Comput. 1, 7 p. (2018)
Arriaga, O., Valdenegro, M., Plöger, P.: Real-time convolutional neural networks for emotion and gender classification. In: ESANN, pp. 221–226 (2019)
Carrier, P.L., Courville, A., Goodfellow, I.J., Mirza, M., Bengio, Y.: Fer-2013 face database. Technical report (2013)
Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: CVPR, vol. 1, pp. 886–893 (2005)
Fasel, B., Luettin, J.: Automatic facial expression analysis: a survey. Pattern Recogn. 36(1), 259–275 (2003)
Hosseini, S., Cho, N.I.: Gf-CapsNet: Using Gabor jet and capsule networks for facial age, gender, and expression recognition. In: FG, pp. 1–8 (2019)
Ioffe, S., Szegedy, C.: Batch normalization: accelerating deep network training by reducing internal covariate shift. In: ICML, ICML 2015, vol. 37, pp. 448–456. JMLR.org (2015)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization (2014)
Ko, B.C.: A brief review of facial emotion recognition based on visual information. Sensors 18(2), 401 (2018)
Liu, C., Wechsler, H.: Gabor feature based classification using the enhanced fisher linear discriminant model for face recognition. IEEE TIP 11(4), 467–476 (2002)
Liu, P., Han, S., Meng, Z., Tong, Y.: Facial expression recognition via a boosted deep belief network. In: CVPR, pp. 1805–1812 (2014)
Lopes, A.T., de Aguiar, E., Souza, A.F.D., Oliveira-Santos, T.: Facial expression recognition with convolutional neural networks: coping with few data and the training sample order. Pattern Recogn. 61, 610–628 (2017)
Marrero Fernandez, P.D., Guerrero Pena, F.A., Ing Ren, T., Cunha, A.: FERAtt: facial expression recognition with attention net. In: CVPR Workshops, pp. 1–10 (2019)
Minaee, S., Abdolrashidi, A.: Deep-emotion: facial expression recognition using attentional convolutional network. CoRR abs/1902.01019 (2019)
Mollahosseini, A., Chan, D., Mahoor, M.H.: Going deeper in facial expression recognition using deep neural networks. In: WACV, pp. 1–10 (2016)
Pramerdorfer, C., Kampel, M.: Facial expression recognition using convolutional neural networks:state of the art. CoRR abs/1612.02903 (2016)
Sabour, S., Frosst, N., Hinton, G.E.: Dynamic routing between capsules. In: NIPS, pp. 3856–3866 (2017)
Tang, Y.: Deep learning using linear support vector machines. In: ICML, pp. 1–6 (2013)
Tariq, U., Yang, J., Huang, T.S.: Multi-view facial expression recognition analysis with generic sparse coding feature. In: Fusiello, A., Murino, V., Cucchiara, R. (eds.) ECCV 2012. LNCS, vol. 7585, pp. 578–588. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-33885-4_58
Viola, P., Jones, M., et al.: Rapid object detection using a boosted cascade of simple features. In: CVPR(1), vol. 1, no. 511–518, p. 3 (2001)
Yu, Z., Zhang, C.: Image based static facial expression recognition with multiple deep network learning. In: ICMI, ICMI 2015, pp. 435–442. ACM, New York (2015)
Zeng, N., Zhang, H., Song, B., Liu, W., Li, Y., Dobaie, A.M.: Facial expression recognition via learning deep sparse autoencoders. Neurocomputing 273, 643–649 (2018)
Zhang, F., Zhang, T., Mao, Q., Xu, C.: Joint pose and expression modeling for facial expression recognition. In: CVPR, pp. 3359–3368 (2018)
Zhang, T., Zheng, W., Cui, Z., Zong, Y., Yan, J., Yan, K.: A deep neural network-driven feature learning method for multi-view facial expression recognition. IEEE Trans. Multimed. 18(12), 2528–2536 (2016)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Shah, B., Bhatt, K., Mandal, S., Mitra, S.K. (2020). EMOTIONCAPS - Facial Emotion Recognition Using Capsules. In: Yang, H., Pasupa, K., Leung, A.CS., Kwok, J.T., Chan, J.H., King, I. (eds) Neural Information Processing. ICONIP 2020. Communications in Computer and Information Science, vol 1332. Springer, Cham. https://doi.org/10.1007/978-3-030-63820-7_45
Download citation
DOI: https://doi.org/10.1007/978-3-030-63820-7_45
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-63819-1
Online ISBN: 978-3-030-63820-7
eBook Packages: Computer ScienceComputer Science (R0)