doi:10.1016/S0167-8655(01)00108-8
Copyright © 2002 Elsevier Science B.V. All rights reserved.
Using moment invariants and HMM in facial expression recognition
Department of Electrical Engineering, National University of Singapore, 10 Kent Ridge Crescent, Singapore 119260, Singapore
Received 16 June 2000;
Revised 31 May 2001.
Available online 27 November 2001.
References and further reading may be available for this article. To view references and further reading you must
purchase this article.
Abstract
Moment invariants are invariant under shifting, scaling and rotation. They are widely used in pattern recognition because of their discrimination power and robustness. HMM method is a natural and highly reliable way of recognition. In this paper, we have proposed a method of using moment invariants as features and HMM as recognition method in facial expression recognition. Sequences of four universal expressions, i.e. anger, disgust, happiness and surprise, are recognized. We were able to attain an accuracy as high as 96.77%.
Author Keywords: Moment invariant; HMM; Pattern recognition; Facial expression; Recognition
Fig. 1. Sample frames of four different expressions used in the recognition.
Fig. 2. The block diagram of the facial expression recognition system.
Fig. 3. Definition of feature extraction areas: the digits are the numbers of the areas.
Fig. 4. Samples of a happy sequence and a disgust sequence (the upper two rows are of a happy equence and the lower two rows are of a disgust sequence).
Fig. 5. Plots of modified feature vector elements of a happy sequence and a disgust sequence (I1, (*); I2, (+); I3, (×); I4, (−): (a) is of a happy sequence and (b) is of a disgust sequence).
Fig. 6. Structure of the HMM used for facial expression recognition results using modified moment invariants J1, J2, J3 and J4 or J1, J2, J4 or J1, J3, J4 or I1, I4, J1−I1, J4−I4.
Table 1. Recognition results using moment invariants I1, I2, I3 and I4

Table 2. Recognition results using modified moment invariants J1, J2, J3 and J4 or J1, J2, J4 or J1, J3, J4 or I1, I4, J1−I1, J4−I4

Table 3. Recognition results only using J1, J4
