Ruofei Du
Liang He
ABSTRACT
We present VRSurus, a smart device designed to recognize the puppeteer's gestures and render tactile feedback to enhance the interactivity of physical puppets in virtual reality (VR). VRSurus is wireless, self-contained, and small enough to be mounted upon any physical puppets. Using machine-learning techniques, VRSurus is able to recognize three gestures: swiping, shaking and thrusting. Actuators (e.g., solenoids, servos and vibration motors) assist with the puppetry visible to the audience and provide tactile feedback on the puppeteer's forearm. As a proof of concept, we implemented a tangible serious VR game using VRSurus that aimed at inspiring children to protect the environment and demonstrated it at the ACM UIST 2015 Student Innovation Contest. Our 3D models, circuitry and the source code are publicly available at www.vrsurus.com.
- Connelly Barnes, David E Jacobs, Jason Sanders, Dan B Goldman, Szymon Rusinkiewicz, Adam Finkelstein, and Maneesh Agrawala. 2008. Video Puppetry: A Performative Interface for Cutout Animation. ACM Transactions on Graphics (TOG) 27, 5 (2008), 124:1-124:10. Google Scholar
Digital Library
- Olivier Bau and Ivan Poupyrev. 2012. REVEL: Tactile Feedback Technology for Augmented Reality. ACM Transactions on Graphics (TOG) 31, 4 (2012), 1-11. Google ScholarDigital Library
- Ari Y Benbasat and Joseph A Paradiso. 2002. An Inertial Measurement Framework for Gesture Recognition and Applications. In Gesture and Sign Language in Human-Computer Interaction. Springer, 9-20. Google ScholarDigital Library
- Keywon Chung, Michael Shilman, Chris Merrill, and Hiroshi Ishii. 2010. OnObject: Gestural Play With Tagged Everyday Objects. In Adjunct Proceedings of the 23nd Annual ACM Symposium on User Interface Software and Technology (UIST). ACM, 379-380. Google ScholarDigital Library
- Mark Hall, Eibe Frank, Geoffrey Holmes, Bernhard Pfahringer, Peter Reutemann, and Ian H Witten. 2009. The WEKA Data Mining Software: An Update. ACM SIGKDD Explorations Newsletter 11, 1 (2009), 10-18. Google ScholarDigital Library
- Robert Held, Ankit Gupta, Brian Curless, and Maneesh Agrawala. 2012. 3D Puppetry: A Kinect-Based Interface for 3D Animation. In Proceedings of the 25th Annual ACM Symposium on User Interface Software & Technology (UIST). ACM, 423-434. Google ScholarDigital Library
- Alexander Kron and Günther Schmidt. 2003. Multi-Fingered Tactile Feedback From Virtual and Remote Environments. In 11th Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems. 16-23. Google ScholarDigital Library
- Kevin A Li, Patrick Baudisch, William G Griswold, and James D Hollan. 2008. Tapping and Rubbing: Exploring New Dimensions of Tactile Feedback With Voice Coil Motors. In Proceedings of the 21st Annual ACM Symposium on User Interface Software and Technology (UIST). ACM, 181-190. Google ScholarDigital Library
- Pedro Lopes, Alexandra Ion, and Patrick Baudisch. 2015. Impacto: Simulating Physical Impact by Combining Tactile Stimulation with Electrical Muscle Stimulation. In Proceedings of the 28th Annual ACM Symposium on User Interface Software & Technology (UIST). ACM, 11-19. Google ScholarDigital Library
- Jesús Ibáñez Martínez. 2014. emoPuppet: Low-Cost Interactive Digital-Physical Puppets with Emotional Expression. In Proceedings of the 11th Conference on Advances in Computer Entertainment Technology. ACM, 44:1-44:4. Google ScholarDigital Library
- Ali Mazalek, Sanjay Chandrasekharan, Michael Nitsche, Tim Welsh, Paul Clifton, Andrew Quitmeyer, Firaz Peer, Friedrich Kirschner, and Dilip Athreya. 2011. I'm in the Game: Embodied Puppet Interface Improves Avatar Control. In Proceedings of the Fifth International Conference on Tangible, Embedded, and Embodied Interaction (TEI). ACM, 129-136. Google ScholarDigital Library
- Ali Mazalek and Michael Nitsche. 2007. Tangible Interfaces for Real-time 3D Virtual Environments. In Proceedings of the International Conference on Advances in Computer Entertainment Technology. ACM, 155-162. Google ScholarDigital Library
- Sushmita Mitra and Tinku Acharya. 2007. Gesture Recognition: A Survey. IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews 37, 3 (2007), 311-324. Google ScholarDigital Library
- Kouichi Murakami and Hitomi Taguchi. 1991. Gesture Recognition Using Recurrent Neural Networks. In Proceedings of the SIGCHI conference on Human factors in computing systems. ACM, 237-242. Google ScholarDigital Library
- Emil Polyak. 2012. Virtual Impersonation Using Interactive Glove Puppets. In SIGGRAPH Asia 2012 Posters. ACM, 31. Google ScholarDigital Library
- Zhou Ren, Junsong Yuan, and Zhengyou Zhang. 2011. Robust Hand Gesture Recognition Based on Finger-Earth Mover's Distance with a Commodity Depth Camera. In Proceedings of the 19th ACM International Conference on Multimedia (MM). ACM, 1093-1096. Google ScholarDigital Library
- Dominik Schmidt, Robert Kovacs, Vikram Mehta, Udayan Umapathi, Sven Köhler, Lung-Pan Cheng, and Patrick Baudisch. 2015. Level-Ups: Motorized Stilts that Simulate Stair Steps in Virtual Reality. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI). ACM, 2157-2160. Google ScholarDigital Library
- Ronit Slyper, Guy Hoffman, and Ariel Shamir. 2015. Mirror Puppeteering: Animating Toy Robots in Front of a Webcam. In Proceedings of the Ninth International Conference on Tangible, Embedded, and Embodied Interaction (TEI). 241-248. Google ScholarDigital Library
- John Weissmann and Ralf Salomon. 1999. Gesture Recognition for Virtual Reality Applications Using Data Gloves and Neural Networks. In International Joint Conference on Neural Networks (IJCNN), Vol. 3. IEEE, 2043-2046.Google Scholar
- Yupeng Zhang, Teng Han, Zhimin Ren, Nobuyuki Umetani, Xin Tong, Yang Liu, Takaaki Shiratori, and Xiang Cao. 2013. BodyAvatar: Creating Freeform 3D Avatars Using First-person Body Gestures. In Proceedings of the 26th Annual ACM Symposium on User Interface Software and Technology (UIST). ACM, 387-396. Google ScholarDigital Library
Index Terms
- VRSurus: Enhancing Interactivity and Tangibility of Puppets in Virtual Reality
Recommendations
Haptic around: multiple tactile sensations for immersive environment and interaction in virtual reality
VRST '18: Proceedings of the 24th ACM Symposium on Virtual Reality Software and TechnologyIn this paper, we present Haptic Around, a hybrid-haptic feedback system, which utilizes fan, hot air blower, mist creator and heat light to recreate multiple tactile sensations in virtual reality for enhancing the immersive environment and interaction. ...
Infrared bundle adjusting and clustering method for head-mounted display and Leap Motion calibration
Leap Motion has become widely used due to its ability to recognize intuitive hand gestures or accurate finger positions. Attaching a Leap Motion to a virtual reality head-mounted display (VR HMD) is highly interoperable with virtual objects in virtual ...
Head-mounted display with mid-air tactile feedback
VRST '15: Proceedings of the 21st ACM Symposium on Virtual Reality Software and TechnologyVirtual and physical worlds are merging. Currently users of head-mounted displays cannot have unobtrusive tactile feedback while touching virtual objects. We present a mid-air tactile feedback system for head-mounted displays. Our prototype uses the ...
Comments