ABSTRACT
The small screens of smartwatches provide limited space for input tasks. Finger identification is a promising technique to address this problem by associating different functions with different fingers. However, current technologies for finger identification are unavailable or unsuitable for smartwatches. To address this problem, this paper observes that normal smartwatch use takes places with a relatively static pose between the two hands. In this situation, we argue that the touch and angle profiles generated by different fingers on a standard smartwatch touch screen will differ sufficiently to support reliable identification. The viability of this idea is explored in two studies that capture touches in natural and exaggerated poses during tapping and swiping tasks. Machine learning models report accuracies of up to 93% and 98% respectively, figures that are sufficient for many common interaction tasks. Furthermore, the exaggerated poses show modest costs (in terms of time/errors) compared to the natural touches. We conclude by presenting examples and discussing how interaction designs using finger identification can be adapted to the smartwatch form factor.
Supplemental Material
- Sebastian Boring, David Ledo, Xiang 'Anthony' Chen, Nicolai Marquardt, Anthony Tang, and Saul Greenberg. 2012. The fat thumb: using the thumb's contact size for single-handed mobile interaction. In Proceedings of the 14th international conference on Human-computer interaction with mobile devices and services (MobileHCI '12), 39--48. http://dx.doi.org/10.1145/2371574.2371582 Google ScholarDigital Library
- Philipp Ewerling, Alexander Kulik, and Bernd Froehlich. 2012. Finger and hand detection for multitouch interfaces based on maximally stable extremal regions. In Proceedings of the 2012 ACM international conference on Interactive tabletops and surfaces (ITS '12), 173--182. http://dx.doi.org/10.1145/2396636.2396663 Google ScholarDigital Library
- Emilien Ghomi, Stéphane Huot, Olivier Bau, Michel Beaudouin-Lafon, and Wendy E. Mackay. 2013. Arpège: learning multitouch chord gestures vocabularies. In Proceedings of the 2013 ACM international conference on Interactive tabletops and surfaces (ITS '13), 209--218. http://dx.doi.org/10.1145/2512349.2512795 Google ScholarDigital Library
- Alix Goguey, Mathieu Nancel, Géry Casiez, and Daniel Vogel. 2016. The Performance and Preference of Different Fingers and Chords for Pointing, Dragging, and Object Transformation. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16), 4250--4261. http://dx.doi.org/10.1145/2858036.2858194 Google ScholarDigital Library
- Aakar Gupta and Ravin Balakrishnan. 2016. DualKey: Miniature Screen Text Entry via Finger Identification. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16), 59--70. http://dx.doi.org/10.1145/2858036.2858052 Google ScholarDigital Library
- Mark Hall, Eibe Frank, Geoffrey Holmes, Bernhard Pfahringer, Peter Reutemann, and Ian H. Witten. 2009. The WEKA data mining software: an update. SIGKDD Explor. Newsl. 11, 1 (November 2009), 10--18. http://dx.doi.org/10.1145/1656274.1656278 Google ScholarDigital Library
- Chris Harrison, Julia Schwarz, and Scott E. Hudson. 2011. TapSense: enhancing finger interaction on touch surfaces. In Proceedings of the 24th annual ACM symposium on User interface software and technology (UIST '11), 627--636. http://dx.doi.org/10.1145/2047196.2047279 Google ScholarDigital Library
- Christian Holz and Patrick Baudisch. 2010. The generalized perceived input point model and how to double touch accuracy by extracting fingerprints. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '10), 581--590. http://dx.doi.org/10.1145/1753326.1753413 Google ScholarDigital Library
- Christian Holz and Patrick Baudisch. 2013. Fiberio: a touchscreen that senses fingerprints. In Proceedings of the 26th annual ACM symposium on User interface software and technology (UIST '13), 41--50. http://dx.doi.org/10.1145/2501988.2502021 Google ScholarDigital Library
- Christian Holz, Senaka Buthpitiya, and Marius Knaust. 2015. Bodyprint: Biometric User Identification on Mobile Devices Using the Capacitive Touchscreen to Scan Body Parts. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15), 3011--3014. http://dx.doi.org/10.1145/2702123.2702518 Google ScholarDigital Library
- Benjamin Lafreniere, Carl Gutwin, Andy Cockburn, and Tovi Grossman. 2016. Faster Command Selection on Touchscreen Watches. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16), 4663--4674. http://dx.doi.org/10.1145/2858036.2858166 Google ScholarDigital Library
- Gierad Laput, Robert Xiao, Xiang 'Anthony' Chen, Scott E. Hudson, and Chris Harrison. 2014. Skin buttons: cheap, small, low-powered and clickable fixed-icon laser projectors. In Proceedings of the 27th annual ACM symposium on User interface software and technology (UIST '14), 389--394. http://dx.doi.org/10.1145/2642918.2647356 Google ScholarDigital Library
- Luis A. Leiva, Alireza Sahami, Alejandro Catala, Niels Henze, and Albrecht Schmidt. 2015. Text Entry on Tiny QWERTY Soft Keyboards. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15), 669--678. http://dx.doi.org/10.1145/2702123.2702388 Google ScholarDigital Library
- I. Scott MacKenzie and R. William Soukoreff. 2003. Phrase sets for evaluating text entry techniques. In CHI '03 Extended Abstracts on Human Factors in Computing Systems (CHI EA '03), 754--755. http://dx.doi.org/10.1145/765891.765971 Google ScholarDigital Library
- Nicolai Marquardt, Johannes Kiemer, and Saul Greenberg. 2010. What caused that touch? expressive interaction with a surface through fiduciary-tagged gloves. In ACM International Conference on Interactive Tabletops and Surfaces (ITS '10), 139--142. http://dx.doi.org/10.1145/1936652.1936680 Google ScholarDigital Library
- Ian Oakley and Doyoung Lee. 2014. Interaction on the edge: offset sensing for small devices. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '14), 169--178. http://dx.doi.org/10.1145/2556288.2557138 Google ScholarDigital Library
- Ian Oakley, DoYoung Lee, M. D. Rasel Islam, and Augusto Esteves. 2015. Beats: Tapping Gestures for Smart Watches. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15), 1237--1246. http://dx.doi.org/10.1145/2702123.2702226 Google ScholarDigital Library
- Ian Oakley, Carina Lindahl, Khanh Le, DoYoung Lee, and MD. Rasel Islam. 2016. The Flat Finger: Exploring Area Touches on Smartwatches. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16), 4238--4249. http://dx.doi.org/10.1145/2858036.2858179 Google ScholarDigital Library
- Simon T. Perrault, Eric Lecolinet, James Eagan, and Yves Guiard. 2013. Watchit: simple gestures and eyesfree interaction for wristwatches and bracelets. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '13). 1451--1460. http://doi.acm.org/10.1145/2470654.2466192Google ScholarDigital Library
- Simon Rogers, John Williamson, Craig Stewart, and Roderick Murray-Smith. 2011. AnglePose: robust, precise capacitive touch tracking via 3d orientation estimation. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '11), 2575--2584. http://doi.acm.org/10.1145/1978942.1979318Google ScholarDigital Library
- Quentin Roy, Yves Guiard, Gilles Bailly, Éric Lecolinet, and Olivier Rioul. "Glass+ skin: An empirical evaluation of the added value of finger identification to basic single-touch interaction on touch screens." In Human-Computer Interaction, pp. 55--71. Springer International Publishing, 2015. http://dx.doi.org/10.1007/978-3-319-22723-8_5 Google ScholarDigital Library
- T. Scott Saponas, Desney S. Tan, Dan Morris, Jim Turner, and James A. Landay. 2010. Making musclecomputer interfaces more practical. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '10), 851--854. http://dx.doi.org/10.1145/1753326.1753451 Google ScholarDigital Library
- Craig Stewart, Michael Rohs, Sven Kratz, and Georg Essl. 2010. Characteristics of pressure-based input for mobile devices. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '10), 801--810. http://dx.doi.org/10.1145/1753326.1753444 Google ScholarDigital Library
- Atsushi Sugiura and Yoshiyuki Koseki. 1998. A user interface using fingerprint recognition: holding commands and data objects on fingers. In Proceedings of the 11th annual ACM symposium on User interface software and technology (UIST '98). ACM, NY, NY, USA, 71--79. http://dx.doi.org/10.1145/288392.288575 Google ScholarDigital Library
- Jingtao Wang and John Canny. 2004. FingerSense: augmenting expressiveness to physical pushing button by fingertip identification. In CHI '04 Extended Abstracts on Human Factors in Computing Systems (CHI EA '04), 1267--1270. http://dx.doi.org/10.1145/985921.986040 Google ScholarDigital Library
- Feng Wang, Xiang Cao, Xiangshi Ren, and Pourang Irani. 2009. Detecting and leveraging finger orientation for interaction with direct-touch surfaces. In Proceedings of the 22nd annual ACM symposium on User interface software and technology (UIST '09), 2332. http://doi.acm.org/10.1145/1622176.1622182Google ScholarDigital Library
- Martin Weigel, Tong Lu, Gilles Bailly, Antti Oulasvirta, Carmel Majidi, and Jürgen Steimle. 2015. iSkin: Flexible, Stretchable and Visually Customizable On-Body Touch Sensors for Mobile Computing. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15), 2991--3000. http://dx.doi.org/10.1145/2702123.2702391 Google ScholarDigital Library
- Hongyi Wen, Julian Ramos Rojas, and Anind K. Dey. 2016. Serendipity: Finger Gesture Recognition using an Off-the-Shelf Smartwatch. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16), 3847--3851. http://dx.doi.org/10.1145/2858036.2858466 Google ScholarDigital Library
- Robert Xiao, Julia Schwarz, and Chris Harrison. 2015. Estimating 3D Finger Angle on Commodity Touchscreens. In Proceedings of the 2015 International Conference on Interactive Tabletops & Surfaces (ITS '15), 47--50. http://dx.doi.org/10.1145/2817721.2817737 Google ScholarDigital Library
- Hui-Shyong Yeo, Juyoung Lee, Andrea Bianchi, and Aaron Quigley. 2016. WatchMI: pressure touch, twist and pan gesture input on unmodified smartwatches. In Proceedings of the 18th International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI '16), 394--399. http://dx.doi.org/10.1145/2935334.2935375 Google ScholarDigital Library
- Yang Zhang and Chris Harrison. 2015. Tomo: Wearable, Low-Cost Electrical Impedance Tomography for Hand Gesture Recognition. In Proceedings of the 28th Annual ACM Symposium on User Interface Software & Technology (UIST '15), 167--173. http://dx.doi.org/10.1145/2807442.2807480 Google ScholarDigital Library
- Cheng Zhang, Junrui Yang, Caleb Southern, Thad E. Starner, and Gregory D. Abowd. 2016. WatchOut: extending interactions on a smartwatch with inertial sensing. In Proceedings of the 2016 ACM International Symposium on Wearable Computers (ISWC '16), 136143. http://dx.doi.org/10.1145/2971763.2971775 Google ScholarDigital Library
Index Terms
- TriTap: Identifying Finger Touches on Smartwatches
Recommendations
MagTouch: Robust Finger Identification for a Smartwatch Using a Magnet Ring and a Built-in Magnetometer
CHI '20: Proceedings of the 2020 CHI Conference on Human Factors in Computing SystemsCompleting tasks on smartwatches often requires multiple gestures due to the small size of the touchscreens and the lack of sufficient number of touch controls that are easily accessible with a finger. We propose to increase the number of functions that ...
Investigating the feasibility of finger identification on capacitive touchscreens using deep learning
IUI '19: Proceedings of the 24th International Conference on Intelligent User InterfacesTouchscreens enable intuitive mobile interaction. However, touch input is limited to 2D touch locations which makes it challenging to provide shortcuts and secondary actions similar to hardware keyboards and mice. Previous work presented a wide range of ...
FingMag: Finger Identification Method for Smartwatch
CHI EA '19: Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing SystemsInteracting with a smartwatch is difficult owing to its small touchscreen. A general strategy to overcome the limitations of the small screen is to increase the input vocabulary. A popular approach to do this is to distinguish fingers and assign ...
Comments