Abstract
In this paper, we present a vision-based system that estimates the pose of users as well as the gestures they perform in real time. This system allow users to interact naturally with an application (virtual reality, gaming) or a robot.
The main components of our system are a 3D upper-body tracker, which estimates human body pose in real-time from a stereo sensor and a gesture recognizer, which classifies output from temporal tracker into gesture classes. The main novelty of our system is the bag-of-features representation for temporal sequences. This representation, though simple, proves to be surprisingly powerful and able to implicitly learn sequence dynamics. Based on this representation, a multi-class classifier, treating the bag of features as the feature vector is applied to estimate the corresponding gesture class.
We show with experiments performed on a HCI gesture dataset that our method performs better than state-of-the-art algorithms and has some nice generalization properties. Finally, we describe virtual and real world applications, in which our system was integrated for multimodal interaction.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Agarwal, S., Awan, A.: Learning to detect objects in images via a sparse, part-based representation. IEEE Trans. Pattern Anal. Mach. Intell. 26(11), 1475–1490 (2004)
Andriluka, M., Roth, S., Schiele, B.: People-tracking-by-detection and people-detection-by-tracking. In: IEEE Conference on Computer Vision and Pattern Recognition, 2008 (2008)
Brand, M., Oliver, N., Pentland, A.: Coupled hidden markov models for complex action recognition. In: CVPR (1996)
Bregler, C., Malik, J.: Tracking people with twists and exponential maps. In: CVPR 1998 (1998)
Delamarre, Q., Faugeras, O.D.: 3D articulated models and multi-view tracking with silhouettes. In: Proceedings of ICCV 1999, pp. 716–721 (1999)
Demirdjian, D.: http://people.csail.mit.edu/demirdji/projects/iwall2.htm
Demirdjian, D., Darrell, T.: 3D articulated pose tracking for untethered deictic reference. In: Proceedings of ICMI 2002, Pittsburgh, PA, USA (2002)
Demirdjian, D., Taycher, L., Shakhnarovich, G., Grauman, K., Darrell, T.: Avoiding the streetlight effect: Tracking by exploring likelihood modes. In: IEEE International Conference on Computer Vision, pp. 357–364 (2005)
Dorko, Gy., Schmid, C.: Selection of scale-invariant parts for object class recognition. In: ICCV, vol. 01, p. 634 (2003)
Duda, R., Hart, P., Stork, D.: Pattern Classification, 2nd edn. Ch.4. John Wiley, Chichester (2001)
Felzenszwalb, P., Huttenlocher, D.: Pictorial structures for object recognition. International Journal of Computer Vision 61 (June 2005)
Lafferty, J., McCallum, A., Pereira, F.: Conditional random fields: probabilistic models for segmenting and labelling sequence data. In: ICML (2001)
Laptev, I., Marszalek, M., Schmid, C., Rozenfeld, B.: Learning realistic human actions from movies. In: IEEE Conference on Computer Vision and Pattern Recognition, 2008. CVPR 2008. pp. 1–8 (2008)
Li, H., Greenspan, M.A.: Multi-scale gesture recognition from time-varying contours. In: ICCV, pp. 236–243 (2005)
Oliver, N., Horvitz, E., Garg, A.: Layered representations for human activity recognition. In: Fourth IEEE Int. Conf. on Multimodal Interfaces, pp. 3–8 (2002)
Rabiner, L.R.: A tutorial on hidden markov models and selected applications in speech recognition. Proc. of the IEEE 77, 257–286 (1989)
Seneff, S., Hurley, E., Lau, R., Pao, C., Schmid, P., Zue, V.: Galaxy-ii: A reference architecture for conversational system development. In: ICSLP, vol. 3, pp. 931–934 (1998)
Sminchiesescu, C., Triggs, B.: Kinematic jump processes for monocular 3d human tracking. In: CVPR (2003)
Sminchisescu, C., Kanaujia, A., Li, Z., Metaxas, D.: Conditional models for contextual human motion recognition. In: Int’l Conf. on Computer Vision (2005)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Demirdjian, D., Varri, C. (2009). Recognizing Gestures for Virtual and Real World Interaction. In: Fritz, M., Schiele, B., Piater, J.H. (eds) Computer Vision Systems. ICVS 2009. Lecture Notes in Computer Science, vol 5815. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-04667-4_1
Download citation
DOI: https://doi.org/10.1007/978-3-642-04667-4_1
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-04666-7
Online ISBN: 978-3-642-04667-4
eBook Packages: Computer ScienceComputer Science (R0)