Abstract
When we encounter an English word that we do not understand, we can look it up in a dictionary. However, when an American Sign Language (ASL) user encounters an unknown sign, looking up the meaning of that sign is not a straightforward process. It has been recently proposed that this problem can be addressed using a computer vision system that helps users look up the meaning of a sign. In that approach, sign lookup can be treated as a video database retrieval problem. When the user encounters an unknown sign, the user provides a video example of that sign as a query, so as to retrieve the most similar signs in the database. A necessary component of such a sign lookup system is a similarity measure for comparing sign videos. Given a query video of a specific sign, the similarity measure should assign high similarity values to videos from the same sign, and low similarity values to videos from other signs. This paper evaluates a state-of-the-art video-based similarity measure called Dynamic Space-Time Warping (DSTW) for the purposes of sign retrieval. The paper also discusses how to specifically adapt DSTW so as to tolerate differences in translation and scale.
Chapter PDF
Similar content being viewed by others
Keywords
References
Alon, J., Athitsos, V., Yuan, Q., Sclaroff, S.: Simultaneous localization and recognition of dynamic hand gestures. In: IEEE Motion Workshop, pp. 254–260 (2005)
Athitsos, V., Neidle, C., Sclaroff, S., Nash, J., Stefan, A., Yuan, Q., Thangali, A.: The American Sign Language lexicon video dataset. In: IEEE Workshop on Computer Vision and Pattern Recognition for Human Communicative Behavior Analysis, CVPR4HB (2008)
Bauer, B., Kraiss, K.: Towards an automatic sign language recognition system using subunits. In: Gesture Workshop, pp. 64–75 (2001)
Chen, F., Fu, C., Huang, C.: Hand gesture recognition using a real-time tracking method and Hidden Markov Models. Image and Video Computing 21(8), 745–758 (2003)
Corradini, A.: Dynamic time warping for off-line recognition of a small gesture vocabulary. In: Recognition, Analysis and Tracking of Faces and Gestures in Real-time Systems (RATFG-RTS), pp. 82–89 (2001)
Cui, Y., Weng, J.: Appearance-based hand sign recognition from intensity image sequences. Computer Vision and Image Understanding 78(2), 157–176 (2000)
Darrell, T., Essa, I., Pentland, A.: Task-specific gesture analysis in real-time using interpolated views. IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI) 18(12), 1236–1242 (1996)
Fujimura, K., Liu, X.: Sign recognition using depth image streams. In: Automatic Face and Gesture Recognition, pp. 381–386 (2006)
Gao, W., Fang, G., Zhao, D., Chen, Y.: Transition movement models for large vocabulary continuous sign language recognition. In: Automatic Face and Gesture Recognition, pp. 553–558 (2004)
Kadir, T., Bowden, R., Ong, E., Zisserman, A.: Minimal training, large lexicon, unconstrained sign language recognition. In: British Machine Vision Conference (BMVC), vol. I
Kruskall, J.B., Liberman, M.: The symmetric time warping algorithm: From continuous to discrete. In: Time Warps. Addison-Wesley, Reading (1983)
Ma, J., Gao, W., Wu, J., Wang, C.: A continuous Chinese Sign Language recognition system. In: Automatic Face and Gesture Recognition, pp. 428–433 (2000)
Ong, S.C.W., Ranganath, S.: Automatic sign language analysis: A survey and the future beyond lexical meaning. IEEE Transactions on Pattern Analysis and Machine Intelligence 27(6), 873–891 (2005)
Rabiner, L.: A tutorial on hidden markov models and selected applications in speech recognition. Proceedings of the IEEEÂ 77(2) (1989)
Rowley, H., Baluja, S., Kanade, T.: Rotation invariant neural network-based face detection. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 38–44 (1998)
Sagawa, H., Takeuchi, M.: A method for recognizing a sequence of sign language words represented in a Japanese Sign Language sentence. In: Automatic Face and Gesture Recognition, pp. 434–439 (2000)
Starner, T., Pentland, A.: Real-time American Sign Language recognition using desk and wearable computer based video. IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI) 20(12), 1371–1375 (1998)
Valli, C. (ed.): The Gallaudet Dictionary of American Sign Language. Gallaudet U. Press, Washington (2006)
Vogler, C., Metaxas, D.N.: Parallel hidden markov models for american sign language recognition. In: IEEE International Conference on Computer Vision (ICCV), pp. 116–122 (1999)
Vogler, C., Metaxas, D.N.: Handshapes and movements: Multiple-channel american sign language recognition. In: Gesture Workshop, pp. 247–258 (2003)
Wang, C., Shan, S., Gao, W.: An approach based on phonemes to large vocabulary Chinese Sign Language recognition. In: Automatic Face and Gesture Recognition, pp. 411–416 (2002)
Yang, M., Ahuja, N.: Recognizing hand gesture using motion trajectories. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), vol. 1, pp. 466–472 (1999)
Yao, G., Yao, H., Liu, X., Jiang, F.: Real time large vocabulary continuous sign language recognition based on OP/Viterbi algorithm. In: International Conference on Pattern Recognition (ICPR), vol. 3, pp. 312–315 (2006)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Wang, H., Stefan, A., Athitsos, V. (2009). A Similarity Measure for Vision-Based Sign Recognition. In: Stephanidis, C. (eds) Universal Access in Human-Computer Interaction. Applications and Services. UAHCI 2009. Lecture Notes in Computer Science, vol 5616. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-02713-0_64
Download citation
DOI: https://doi.org/10.1007/978-3-642-02713-0_64
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-02712-3
Online ISBN: 978-3-642-02713-0
eBook Packages: Computer ScienceComputer Science (R0)