Chun Yu
ABSTRACT
Despite the increasing popularity of head mounted displays (HMDs), development of efficient text entry methods on these devices has remained under explored. In this paper, we investigate the feasibility of head-based text entry for HMDs, by which, the user controls a pointer on a virtual keyboard using head rotation. Specifically, we investigate three techniques: TapType, DwellType, and GestureType. Users of TapType select a letter by pointing to it and tapping a button. Users of DwellType select a letter by pointing to it and dwelling over it for a period of time. Users of GestureType perform word-level input using a gesture typing style. Two lab studies were conducted. In the first study, users typed 10.59 WPM, 15.58 WPM, and 19.04 WPM with DwellType, TapType, and GestureType, respectively. Users subjectively felt that all three of the techniques were easy to learn and considered the induced fatigue to be acceptable. In the second study, we further investigated GestureType. We improved its gesture-word recognition algorithm by incorporating the head movement pattern obtained from the first study. This resulted in users reaching 24.73 WPM after 60 minutes of training. Based on these results, we argue that head-based text entry is feasible and practical on HMDs, and deserves more attention.
Supplemental Material
- 2011. The Open American National Corpus. http://www.anc.org/Google Scholar
- Ouais Alsharif, Tom Ouyang, Francçoise Beaufays, Shumin Zhai, Thomas Breuel, Johan Schalkwyk. Long short term memory neural network for keyboard gesture decoding. In 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2015. 2076--2080Google Scholar
- Christoph Amma, Marcus Georgi, and Tanja Schultz. 2012. Hands-Free Mobile Text Input by Spotting and Continuous Recognition of 3d-Space Handwriting with Inertial Sensors. In 2012 16th International Symposium on Wearable Computers (ISWC'12), 52--59. Google ScholarDigital Library
- Xiaojun Bi, Ciprian Chelba, Tom Ouyang, Kurt Partridge, and Shumin Zhai. 2012. Bimanual gesture keyboard. In Proceedings of the 25th annual ACM symposium on User interface software and technology (UIST '12). 137--146. Google ScholarDigital Library
- Doug A. Bowman, Vinh Q. Ly, and Joshua M. Campbell. 2001. Pinch keyboard: Natural text input for immersive virtual environments.Google Scholar
- Xiang 'Anthony' Chen, Tovi Grossman, and George Fitzmaurice. 2014. Swipeboard: a text entry technique for ultra-small interfaces that supports novice to expert transitions. In Proceedings of the 27th annual ACM symposium on User interface software and technology (UIST '14). 615--620. Google ScholarDigital Library
- Weiya Chen, Anthony Plancoulaine, Nicolas Férey, Damien Touraine, Julien Nelson, and Patrick Bourdot. 2013. 6DoF navigation in virtual worlds: comparison of joystick-based and head-controlled paradigms. In Proceedings of the 19th ACM Symposium on Virtual Reality Software and Technology(VRST '13). 111--114. Google ScholarDigital Library
- Wenxin Feng, Ming Chen, and Margrit Betke. 2014. Target reverse crossing: a selection method for camerabased mouse-replacement systems. In Proceedings of the 7th International Conference on PErvasive Technologies Related to Assistive Environments (PETRA '14). Article 39, 4 pages. Google ScholarDigital Library
- Yulia Gizatdinova, Oleg Špakov, and Veikko Surakka. 2012. Comparison of video-based pointing and selection techniques for hands-free text entry. In Proceedings of the International Working Conference on Advanced Visual Interfaces (AVI '12), Genny Tortora, Stefano Levialdi, and Maurizio Tucci (Eds.). 132--139. Google ScholarDigital Library
- Joshua Goodman, Gina Venolia, Keith Steury, and Chauncey Parker. 2002. Language modeling for soft keyboards. In Proceedings of the 7th international conference on Intelligent user interfaces (IUI '02). 194195. Google ScholarDigital Library
- Mitchell Gordon, Tom Ouyang, and Shumin Zhai. 2016. WatchWriter: Tap and Gesture Typing on a Smartwatch Miniature Keyboard with Statistical Decoding. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16). 3817--3821. Google ScholarDigital Library
- Tovi Grossman, Xiang Anthony Chen, and George Fitzmaurice. 2015. Typing on Glasses: Adapting Text Entry to Smart Eyewear. In Proceedings of the 17th International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI '15). 144--152. Google ScholarDigital Library
- John Paulin Hansen, Kristian Tørning, Anders Sewerin Johansen, Kenji Itoh, and Hirotaka Aoki. 2004. Gaze typing compared with input by head and hand. In Proceedings of the 2004 symposium on Eye tracking research & applications (ETRA '04). 131--138. Google ScholarDigital Library
- Per Ola Kristensson and Keith Vertanen. 2012. The potential of dwell-free eye-typing for fast assistive gaze communication. In Proceedings of the Symposium on Eye Tracking Research and Applications (ETRA '12), Stephen N. Spencer (Ed.). 241--244. Google ScholarDigital Library
- Per Ola Kristensson and Shumin Zhai. 2004. SHARK2 : a large vocabulary shorthand writing system for penbased computers. In Proceedings of the 17th annual ACM symposium on User interface software and technology (UIST '04). 43--52. Google ScholarDigital Library
- Andrew Kurauchi, Wenxin Feng, Ajjen Joshi, Carlos Morimoto, and Margrit Betke. 2016. EyeSwipe: Dwellfree Text Entry Using Gaze Paths. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16). 1952--1956. Google ScholarDigital Library
- I. Scott MacKenzie and R. William Soukoreff. 2002. A character-level error analysis technique for evaluating text entry methods. In Proceedings of the second Nordic conference on Human-computer interaction (NordiCHI '02). 243--246. Google ScholarDigital Library
- I. Scott MacKenzie and R. William Soukoreff. 2003. Phrase sets for evaluating text entry techniques. In CHI '03 Extended Abstracts on Human Factors in Computing Systems (CHI EA '03). 754--755. Google ScholarDigital Library
- Päivi Majaranta and Kari-Jouko Räihä. 2007. Text entry by gaze: Utilizing eye-tracking. Text entry systems: Mobility, accessibility, universality: 175--187.Google Scholar
- Päivi Majaranta, Ulla-Kaija Ahola, and Oleg Špakov. 2009. Fast gaze typing with an adjustable dwell time. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '09). 357--360. Google ScholarDigital Library
- Anders Markussen, Mikkel R. Jakobsen, and Kasper Hornbæk. 2013. Selection-based mid-air text entry on large displays. In IFIP Conference on HumanComputer Interaction. 401--418.Google ScholarCross Ref
- Anders Markussen, Mikkel Rønne Jakobsen, and Kasper Hornbæk. 2014. Vulture: a mid-air wordgesture keyboard. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '14). 1073--1082. Google ScholarDigital Library
- Roderick McCall, Benoît Martin, Andrei Popleteev, Nicolas Louveton and Thomas Engel. 2015. Text entry on smart glasses. In 2015 8th International Conference on Human System Interaction (HSI). 195--200.Google ScholarCross Ref
- Tao Ni, Doug Bowman, and Chris North. 2011. AirStroke: bringing unistroke text entry to freehand gesture interfaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '11). 2473--2476. Google ScholarDigital Library
- Paul Nation and Robert Waring. 1997. Vocabulary size, text coverage and word lists. Vocabulary: Description, acquisition and pedagogy Rev 14: 6--19.Google Scholar
- Diogo Pedrosa, Maria Da Graça Pimentel, Amy Wright, and Khai N. Truong. 2015. Filteryedping: Design Challenges and User Performance of DwellFree Eye Typing. ACM Trans. Access. Comput. 6, 1, Article 3 (March 2015), 37 pages. Google ScholarDigital Library
- Shyam Reyal, Shumin Zhai, and Per Ola Kristensson. 2015. Performance and User Experience of Touchscreen and Gesture Keyboards in a Lab Setting and in the Wild. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15). 679--688. Google ScholarDigital Library
- Sayan Sarcar, Prateek Panwar, and Tuhin Chakraborty. 2013. EyeK: an efficient dwell-free eye gaze-based text entry system. In Proceedings of the 11th Asia Pacific Conference on Computer Human Interaction (APCHI '13). 215--220. Google ScholarDigital Library
- Alexander Schick, Daniel Morlock, Christoph Amma, Tanja Schultz, and Rainer Stiefelhagen. 2012. Visionbased handwriting recognition for unrestricted text input in mid-air. In Proceedings of the 14th ACM international conference on Multimodal interaction (ICMI '12). 217--220. Google ScholarDigital Library
- Srinath Sridhar, Anna Maria Feit, Christian Theobalt, and Antti Oulasvirta. 2015. Investigating the Dexterity of Multi-Finger Input for Mid-Air Text Entry. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15). 3643--3652. Google ScholarDigital Library
- Keith Vertanen, Haythem Memmi, Justin Emge, Shyam Reyal, and Per Ola Kristensson. 2015. VelociTap: Investigating Fast Mobile Text Entry using Sentence-Based Decoding of Touchscreen Keyboard Input. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15). 659--668. Google ScholarDigital Library
- Daniel Vogel and Patrick Baudisch. 2007. Shift: a technique for operating pen-based interfaces using touch. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '07). 657666. Google ScholarDigital Library
- Xin Yi, Chun Yu, Mingrui Zhang, Sida Gao, Ke Sun, and Yuanchun Shi. 2015. ATK: Enabling Ten-Finger Freehand Typing in Air Based on 3D Hand Tracking Data. In Proceedings of the 28th Annual ACM Symposium on User Interface Software & Technology (UIST '15). 539--548. Google ScholarDigital Library
- Chun Yu, Ke Sun, Mingyuan Zhong, Xincheng Li, Peijun Zhao, and Yuanchun Shi. 2016. OneDimensional Handwriting: Inputting Letters and Words on Smart Glasses. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16). 71--82. Google ScholarDigital Library
- Shumin Zhai, Per Ola Kristensson, Pengjun Gong, Michael Greiner, Shilei Allen Peng, Liang Mico Liu, and Anthony Dunnigan. 2009. Shapewriter on the iphone: from the laboratory to the real world. InCHI '09 Extended Abstracts on Human Factors in Computing Systems (CHI EA '09). 2667--2670. Google ScholarDigital Library
- Shumin Zhai, Carlos Morimoto, and Steven Ihde. 1999. Manual and gaze input cascaded (MAGIC) pointing. In Proceedings of the SIGCHI conference on Human Factors in Computing Systems (CHI '99). 246--253. Google ScholarDigital Library
- Shumin Zhai and Per-Ola Kristensson. 2003. Shorthand writing on stylus keyboard. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '03). 97--104. Google ScholarDigital Library
- Shumin Zhai and Per Ola Kristensson. 2012. The word-gesture keyboard: reimagining keyboard interaction. Commun. ACM 55, 9 (September 2012): 91--101. Google ScholarDigital Library
Index Terms
- Tap, Dwell or Gesture?: Exploring Head-Based Text Entry Techniques for HMDs
Recommendations
Investigating Tilt-based Gesture Keyboard Entry for Single-Handed Text Entry on Large Devices
CHI '17: Proceedings of the 2017 CHI Conference on Human Factors in Computing SystemsThe popularity of mobile devices with large screens is making single-handed interaction difficult. We propose and evaluate a novel design point around a tilt-based text entry technique which supports single handed usage. Our technique is based on the ...
Investigating Gesture Typing for Indirect Touch
With the development of ubiquitous computing, entering text on HMDs and smart TVs using handheld touchscreen devices (e.g., smartphone and controller) is becoming more and more attractive. In these indirect touch scenarios, the touch input surface is ...
DRG-Keyboard: Enabling Subtle Gesture Typing on the Fingertip with Dual IMU Rings
We present DRG-Keyboard, a gesture keyboard enabled by dual IMU rings, allowing the user to swipe the thumb on the index fingertip to perform word gesture typing as if typing on a miniature QWERTY keyboard. With dual IMUs attached to the user's thumb and ...
Comments