Abstract
Hand gestures are an efficient manner for human computer interaction (HCI). They can also be used for the development of a non-intrusive biometrics system. In this paper, we address the issues of hand detection and gesture tracking using a single camera. A simple yet effective approach is proposed for applications with complex backgrounds and minimal constraints on the subject. A hand detection approach is presented using a Bayesian classifier based on Gaussian Mixture Models (GMM) for identifying pixels of skin color. A connected component based region-growing algorithm is included for forming areas of skin pixels into areas of likely hand candidates. Given the detected hand region, we further detect the hand features using a deformable model for hand gesture estimation. We propose a novel method, a 3D physics-based dynamic mesh adaptation approach, to estimate and track hand shape and finger directions. The physics-based hand model adaptation algorithm allows us to model hand shape and orientation at the same time, thereby improving the robustness and speed for hand gesture tracking and regeneration.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Cinque, L., et al.: Fast viewpoint-invariant articulated hand detection combining curver and graph matching. In: FGR 2008 (2008)
Chik, D., et al.: Using an Adaptive VAR Model for Motion Prediction in 3D Hand Tracking. In: IEEE FGR 2008 (2008)
Suk, H., Sin, B., Lee, S.: Recognizing Hand Gestures using Dynamic Bayesian Network. In: IEEE FGR 2008 (2008)
Moeslund, T.B., Granum, E.: A survey of computer vision-based human motion capture. Computer Vision and Image Understanding 81(3), 231–268 (2001)
Pavlovic, V., Huang, T., et al.: Visual interpretation of hand gestures for HCI: a review. IEEE Trans. PAMI (1997)
Kakumanu, P., Bourbakis, N., et al.: A survey of skin-color modeling and detection methods. In: Pattern Recognition (2006)
Kurata, T., Okuma, T., Kourogi, M., Sakaue, K.: The hand mouse: Gmm hand-color classification and mean shifttracking. In: IEEE ICCV Workshop on Recognition, Analysis, and Tracking of Faces and Gestures in Real-Time Systems, pp. 119–124 (2001)
Lee, M., Weinshall, D., Cohen-Solal, E., Colmenarez, A., Lyons, D.: A Computer Vision System for On-Screen Item Selection by Finger Pointing. In: IEEE CVPR 1999, vol. 1, p. 1999 (2001)
Manders, C., Farbiz, F., Chong, J., Tang, K., Chua, G., Loke, M., Yuan, M.: Robust hand tracking using a skin tone and depth joint probability model. In: FGR 2008 (2008)
Park, C., Roh, M., Lee, S.: Real-Time 3D Pointing Gesture Recognition in Mobile Space. In: FGR 2008 (2008)
Nguyen, T., Binh, N., Bischof, H.: An active boosting-based learning framework for real-time hand detection. In: FGR 2008 (2008)
CyberGlove, http://www.immersion.com/3d/products/cyber_glove.php
Kolsch, M., Turk, M.: Robust hand detection. In: IEEE FGR 2004 (2004)
Oka, K., Sato, Y., Koike, H.: Real-time fingertip tracking and gesture recognition. IEEE CG&A (2002)
Argyros, A.A., Lourakis, M.I.A.: Real-time tracking of multiple skin-colored objects with a possibly moving camera. In: Pajdla, T., Matas, J(G.) (eds.) ECCV 2004. LNCS, vol. 3023, pp. 368–379. Springer, Heidelberg (2004)
Wu, Y., Huang, T.: Vision-based gesture recognition: A review. In: The 3rd Gesture Workshop (1999)
Kalra, P., Magnenat-Thalmann, N., Mossozet, L., Sannier, G., Aubel, A., Thalmann, D.: Real-time animation of realistic virtual humans. IEEE Computer Graphics and Application, 42–56 (1998)
Lee, M., Lyons, D., et al.: A computer vision system for on-screen item selection by finger pointing. In: CVPR 2001 (2001)
Jojic, N., et al.: Detection and estimation of pointing gestures in real-time stereo sequences. In: IEEE Automatic Face and Gesture Recognition, FGR 2000 (2000)
Yamamoto, Y., et al.: Arm-pointing gesture interface using surrounded stereo cameras system. In: IEEE International Conference on Pattern Recognition (2004)
Colombo, C., et al.: Visual capture and understanding of hand pointing actions in a 3-D environment. IEEE Trans. on SMC-B, 677–686 (2003)
Vezhnevets, V., Sazonov, V., Andreeva, A.: A survey on pixel-based skin color detection techniques. In: Proc. Graphicon, pp. 85–92 (2003)
Bilmes, J.A.: A gentle tutorial of the em algorithm and its application to parameter estimation for gaussian mixture and hidden markov models. Technical Report TR-97-021, International Computer Science Insitute and Computer Science Division U.C. Berkeley, Berkeley CA (1998)
Paalanen, P.: Bayesian classification using gaussian mixcute model and EM estimation: implementation and comparisons. Information Technology Project (2004)
Terzopoulos, D., Vasilescu, M.: Sampling and reconstruction with adaptive meshes. In: IEEE CVPR 1991, pp. 70–75 (1991)
Dadgostar, F., Barczak, A., Sarrafzadeh, A.: Massey hand gesture database, http://www.massey.ac.nz/fdadgost/xview.php?page=hand_image_database/default
Soh, J., Yoon, H.-S., Wang, M., Min, B.-W.: Locating hands in complex images using color analysis. In: IEEE International Conference on Systems, Man, and Cybernetics, pp. 2142–2146 (1997)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Yoder, G., Yin, L. (2009). Real-Time Hand Detection and Gesture Tracking with GMM and Model Adaptation. In: Bebis, G., et al. Advances in Visual Computing. ISVC 2009. Lecture Notes in Computer Science, vol 5876. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-10520-3_36
Download citation
DOI: https://doi.org/10.1007/978-3-642-10520-3_36
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-10519-7
Online ISBN: 978-3-642-10520-3
eBook Packages: Computer ScienceComputer Science (R0)