Liang-Guo Zhang
Wen Gao
ABSTRACT
In this paper, a vision-based medium vocabulary Chinese sign language recognition (SLR) system is presented. The proposed recognition system consists of two modules. In the first module, techniques of robust hands detection, background subtraction and pupils detection are efficiently combined to precisely extract the feature information with the aid of simple colored gloves in the unconstrained environment. Meanwhile, an effective and efficient hierarchical feature description scheme with different scale features to characterize sign language is proposed, where principal component analysis (PCA) is employed to characterize the finger features more elaborately. In the second part, a Tied-Mixture Density Hidden Markov Models (TMDHMM) framework for SLR is proposed, which can speed up the recognition without the significant loss of recognition accuracy compared with the continuous hidden Markov models (CHMM). Experimental results based on 439 frequently used Chinese sign language (CSL) words show that the proposed methods can work well for the medium vocabulary SLR in the environment without special constraints and the recognition accuracy is up to 92.5%.
- R. H. Liang, and M. Ouhyoung, A real-time continuous gesture recognition system for sign language, Proc. 3rd Int'l Conf. Automatic Face and Gesture Recognition, Nara, pp.558--565, 1998.]] Google Scholar
Digital Library
- M.W. Kadous, Machine recognition of Auslan signs using PowerGloves: towards large-lexicon recognition of sign language, Proc. Workshop on the Integration of Gesture in Language and Speech, pp. 165--174, 1996.]]Google Scholar
- S.S. Fels and G.E. Hinton, Glove-talk: A neural network interface between a data-glove and a speech synthesizer, IEEE Trans. Neural Networks, vol. 4, no. 1, pp. 2--8, 1993.]]Google ScholarDigital Library
- J.S. Kim, W. Jang, and Z. Bien, A dynamic gesture recognition system for the Korean sign language (KSL), IEEE Trans. Systems, Man, and Cybernetics, vol. 26, no. 2, pp. 354--359, 1996.]] Google ScholarDigital Library
- W. Gao, J.Y. Ma, J.Q. Wu, and C.L. Wang, Sign language recognition based on HMM/ANN/DP, Int'l J. Pattern Recognition and Artificial Intelligence, vol. 14, no. 5, pp. 587--602, 2000.]]Google ScholarCross Ref
- H. Mastuo, S. Igi, S. Lu, Y. Nagashima, Y. Takata, and T. Teshima, The recognition algorithm with non-contact for Japanese sign language using morphological analysis, Proc. Int'l Gesture Workshop, pp. 273--284, 1997.]] Google ScholarDigital Library
- T. Starner, J. Weaver, and A. Pentland, Real-time American sign language recognition using desk and wearable computer based video, IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 20, no. 12, pp. 1371--1375, 1998.]] Google ScholarDigital Library
- C. Vogler and D. Metaxas, Adapting hidden Markov models for ASL recognition by using three-dimensional computer vision methods, Proc. IEEE Int'l Conf. Systems, Man and Cybernetics, Orlando, pp.156--161, 1997.]]Google ScholarCross Ref
- C. Vogler and D. Metaxas, Toward scalability in ASL recognition: breaking down signs into phonemes, Proc. Int'l Gesture Workshop, Gif-sur-Yvette, France, pp. 400--404, 1999.]] Google ScholarDigital Library
- C. Vogler and D. Metaxas, A framework for recognizing the simultaneous aspects of American sign language, Computer Vision and Image Understanding, vol. 81, no. 3, pp. 358--384, 2001.]] Google ScholarDigital Library
- K. Grobel and M. Assan, Isolated sign language recognition using hidden Markov models, Proc. IEEE Int'l Conf. Systems, Man and Cybernetics, pp. 162--167, 1996.]]Google Scholar
- B. Bauer and H. Hienz, Relevant features for video-based continuous sign language recognition, Proc. 4th Int'l Conf. Automatic Face and Gesture Recognition, Grenoble, pp. 440--445, 2000.]] Google ScholarDigital Library
- L. R. Rabiner, A tutorial on hidden Markov models and selected applications in speech recognition, Proceedings of the IEEE, vol. 77, no. 2, pp. 257--285, 1989.]] Google ScholarCross Ref
- V. V. Digalakis, P. Monaco and H. Murveit, Genones: generalized mixture tying in continuous hidden Markov model-based speech recognizers. IEEE Trans. Speech and Audio Processing, vol 4, no. 4, pp. 281--289, 1996.]]Google ScholarCross Ref
- J. Miao, W. Gao et al, Gravity-Center Template Based Human Face Feature Detection, Proc. 3rd Int'l Conf. Multimodal Interface, Beijing, pp. 207--214, 2000.]] Google ScholarDigital Library
- S. Jabri, Z. Duric, H. Wechsler, and A. Rosenfeld, Detection and location of people in video images using adaptive fusion of color and edge information, Proc. Int'l Conf. Pattern Recognition, Barcelona, Spain, vol. 4, pp. 627--630, 2000.]] Google ScholarDigital Library
- C. Wang and W. Gao, The application of HMMs consisting of different number states in CSL recognition, Journal of Computer Research and Development, vol. 38, no. 1, pp.111--115, 2001 (In Chinese).]]Google Scholar
Index Terms
- A vision-based sign language recognition system using tied-mixture density HMM
Recommendations
Recent developments in visual sign language recognition
Research in the field of sign language recognition has made significant advances in recent years. The present achievements provide the basis for future applications with the objective of supporting the integration of deaf people into the hearing ...
Relevant Features for Video-Based Continuous Sign Language Recognition
FG '00: Proceedings of the Fourth IEEE International Conference on Automatic Face and Gesture Recognition 2000This paper describes the development of a video-based continuous sign language recognition system. The system is based on continuous density Hidden Markov Models (HMM) with one model for each sign. Feature vectors reflecting manual sign parameters serve ...
Coupled HMM-based multi-sensor data fusion for sign language recognition
A novel multi-sensor framework is proposed for Sign Language Recognition (SLR).The framework recognize dynamic sign words performed by hearing impaired persons.A recognition combination framework using Coupled-HMM (CHMM) is proposed for SLR.Results ...
Comments