Philipp Kirschthaler
Martin Porcheron
Joel E. Fischer
ABSTRACT
Discoverability, the ability for users to find and execute features through a user interface, is a recurrent problem with Voice User Interface (VUI) design that makes it difficult for users to understand what commands are supported by a newly encountered system. We studied the effects of two different discoverability strategies proposed in literature, one which provides informational prompts automatically and one which provides help only when the user requests it by asking 'What Can I Say?'. Our study adopted a Wizard of Oz approach that allowed users to order food delivery by voice. Through statistical analysis, we confirmed the beneficial nature of both strategies, with significantly better task performance and higher usability scores in comparison to a baseline. This suggests designers should consider the use of a discoverability strategy in the design of VUIs. While no significant differences were found between the strategies, a majority of the participants highlighted their preference for the 'What Can I Say?' strategy if they were to use the VUI more frequently. Finally, we reflect on the implications for the design of VUIs, highlighting the need to distinguish between initial use and longer-term use in the selection of a strategy.
- Amazon Inc. 2019. Alexa Design Guide. Retrieved 2020-02-13 from https://developer.amazon.com/docs/alexa-design/get-started.htmlGoogle Scholar
- Frank Bentley, Chris Luvogt, Max Silverman, Rushani Wirasinghe, Brooke White, and Danielle Lottridge. 2018. Understanding the Long-Term Use of Smart Speaker Assistants. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 2, 3, Article 91 (9 2018), 24 pages. https://doi.org/10.1145/3264901Google ScholarDigital Library
- Daniel G Bobrow, Ronald M Kaplan, Martin Kay, Donald A Norman, Henry Thompson, and Terry Winograd. 1977. GUS, a frame-driven dialog system. Artificial Intelligence 8, 2 (1977), 155--173. https://doi.org/10.1016/0004-3702(77)90018-2Google ScholarDigital Library
- John Brooke. 1996. SUS: A 'quick and dirty' usability scale. In Usability Evaluation In Industry (1 ed.), Patrick W Jordan, Bruce Thomas, Bernard A Weerdmeester, and Ian L McClelland (Eds.). CRC Press, London, UK, Chapter 21, 189--194.Google Scholar
- Graham Button and Wes Sharrock. 1995. On simulacrums of conversation: Toward a clarification of the relevance of conversation analysis for human-computer interaction. In The Social and Interactional Dimensions of Human-Computer Interface, Pete J Thomas (Ed.). Press Syndicate of the University of Cambridge, Cambridge, UK, Chapter 6, 107--125.Google Scholar
- Hyunji Chung, Michaela Iorga, Jeffrey Voas, and Sangjin Lee. 2017. "Alexa, Can I Trust You?". Computer 50, 9 (2017), 100--104. https://doi.org/10.1109/MC.2017.3571053Google ScholarDigital Library
- Michael H Cohen, James P Giangola, and Jennifer Balogh. 2004. Voice User Interface Design. Addison Wesley Longman Publishing Co., Inc., Redwood City, CA, USA.Google ScholarDigital Library
- Eric Corbett and Astrid Weber. 2016. What can I say? Addressing User Experience Challenges of a Mobile Voice User Interface for Accessibility. In Proceedings of the 18th International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI '16). ACM, New York, NY, USA, 72--82. https://doi.org/10.1145/2935334.2935386Google ScholarDigital Library
- Nils Dahlbäck, Arne Jönsson, and Lars Ahrenberg. 1993. Wizard of Oz studies---Why and how. Knowl.-Based Syst. 6 (12 1993), 258--266. https://doi.org/10.1016/0950--7051(93)90017-NGoogle Scholar
- Michael V Ellis. 1999. Repeated Measures Designs. The Counseling Psychologist 27, 4 (1999), 552--578. https://doi.org/10.1177/0011000099274004Google ScholarCross Ref
- Jinjuan Feng, Clare-Marie Karat, and Andrew Sears. 2004. How productivity improves in hands-free continuous dictation tasks: lessons learned from a longitudinal study. Interacting with Computers 17, 3 (07 2004), 265--289. https://doi.org/10.1016/j.intcom.2004.06.013Google Scholar
- Joel E Fischer, Stuart Reeves, Martin Porcheron, and Rein Ove Sikveland. 2019. Progressivity for Voice Interface Design. In Proceedings of the 1st International Conference on Conversational User Interfaces (CUI '19). ACM, New York, NY, USA, Article 26, 8 pages. https://doi.org/10.1145/3342775.3342788Google ScholarDigital Library
- Norman M Fraser and Nigel Gilbert. 1991. Simulating speech systems. Computer Speech & Language 5, 1 (1991), 81--99. https://doi.org/10.1016/0885--2308(91)90019-MGoogle ScholarCross Ref
- Anushay Furqan, Chelsea Myers, and Jichen Zhu. 2017. Learnability through Adaptive Discovery Tools in Voice User Interfaces. In Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA '17). ACM, New York, NY, USA, 1617--1623.Google ScholarDigital Library
- Debjyoti Ghosh, Pin Sym Foong, Shan Zhang, and Shengdong Zhao. 2018. Assessing the Utility of the System Usability Scale for Evaluating Voice-Based User Interfaces. In Proceedings of the Sixth International Symposium of Chinese CHI (Montreal, QC, Canada) (ChineseCHI '18). Association for Computing Machinery, New York, NY, USA, 11--15. https://doi.org/10.1145/3202667.3204844Google ScholarDigital Library
- Jaakko Hakulinen, Markku Turunen, and Esa-Pekka Salonen. 2005. Software Tutors for Dialogue Systems. In Text, Speech and Dialogue (TSD 2005), Václav Matoušek, Pavel Mautner, and Tomáš Pavelka (Eds.). Springer, Berlin, Germany, 412--419. https://doi.org/10.1007/11551874_53Google ScholarDigital Library
- Ruimin Hu, Shaojian Zhu, Jinjuan Feng, and Andrew Sears. 2011. Use of Speech Technology in Real Life Environment. In Universal Access in Human-Computer Interaction. Applications and Services (UAHCI 11). Springer Berlin Heidelberg, Berlin, Heidelberg, 62--71. https://doi.org/10.1007/978--3-642-21657-2_7Google Scholar
- KAMITIS. 2016. Intelligent Personal Assistant --- Products, Technologies and Market: 2017--2022. Technical Report. KAMITIS, Lyon, France.Google Scholar
- Laura Klein. 2015. Design for Voice Interfaces (1 ed.). O'Reilly Media, Sebastopol, CA, USA.Google Scholar
- Brian Krisler and Richard Alterman. 2008. Training towards Mastery: Overcoming the Active User Paradox. In Proceedings of the 5th Nordic Conference on Human-Computer Interaction: Building Bridges (Lund, Sweden) (NordiCHI '08). ACM, New York, NY, USA, 239--248. https://doi.org/10.1145/1463160.1463186Google ScholarDigital Library
- Gabriel Lyons, Vinh Tran, Carsten Binnig, Ugur Cetintemel, and Tim Kraska. 2016. Making the Case for Query-by-Voice with EchoQuery. In Proceedings of the 2016 International Conference on Management of Data (San Francisco, California, USA) (SIGMOD '16). ACM, New York, NY, USA, 2129--2132. https://doi.org/10.1145/2882903.2899394Google ScholarDigital Library
- Nikolas Martelaro and Wendy Ju. 2017. WoZ Way: Enabling Real-Time Remote Interaction Prototyping & Observation in On-Road Vehicles. In Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing (CSCW '17). ACM, New York, NY, USA, 169--182. https://doi.org/10.1145/2998181.2998293Google ScholarDigital Library
- MeasuringU. 2011. Measuring Usability with the System Usability Scale (SUS). Retrieved 2020-02-13 from https://measuringu.com/sus/Google Scholar
- Chelsea Myers, Anushay Furqan, Jessica Nebolsky, Karina Caro, and Jichen Zhu. 2018. Patterns for How Users Overcome Obstacles in Voice User Interfaces. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (Montreal QC, Canada) (CHI '18). ACM, New York, NY, USA, Article 6, 7 pages. https://doi.org/10.1145/3173574.3173580Google ScholarDigital Library
- Americo Talarico Neto, Renata Pontin M. Fortes, and Adalberto G. da Silva Filho. 2008. Multimodal Interfaces Design Issues: The Fusion of Well-Designed Voice and Graphical User Interfaces. In Proceedings of the 26th Annual ACM International Conference on Design of Communication (SIGDOC '08). ACM, New York, NY, USA, 277--278. https://doi.org/10.1145/1456536.1456597Google Scholar
- Jakob Nielsen. 1994. Enhancing the Explanatory Power of Usability Heuristics. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '94). ACM, New York, NY, USA, 152--158. https://doi.org/10.1145/191666.191729Google ScholarDigital Library
- Cathy Pearl. 2016. Designing Voice User Interfaces: Principles of Conversational Experiences (1 ed.). O'Reilly Media, Sebastopol, CA, USA. https://doi.org/10.2307/4003768Google Scholar
- Martin Porcheron, Joel E Fischer, Stuart Reeves, and Sarah Sharples. 2018. Voice Interfaces in Everyday Life. In Proceedings of the 2018 ACM Conference on Human Factors in Computing Systems (CHI '18). ACM, New York, NY, USA, Article 640, 12 pages. https://doi.org/10.1145/3173574.3174214Google ScholarDigital Library
- Martin Porcheron, Joel E Fischer, and Michel Valstar. 2020. NottReal: A tool for voice-based Wizard of Oz studies. In Proceedings of the 2nd Conference on Conversational User Interfaces (Bilbao, Spain) (CUI '20). ACM, New York, NY, USA, 3. https://doi.org/10.1145/3405755.3406168Google ScholarDigital Library
- Stuart Reeves. 2019. Conversation considered harmful?. In Proceedings of the 1st International Conference on Conversational User Interfaces (CUI '19). ACM, New York, NY, USA, Article 10, 3 pages. https://doi.org/10.1145/3342775.3342796Google ScholarDigital Library
- Dirk Schnelle and Fernando Lyardet. 2006. Voice User Interface Design Patterns. In Eleventh European Conference on Pattern Languages of Programs, (EuroPLoP' 2006). Hillside Europe, Munich, Germany, 27.Google Scholar
- Andrew Sears, Jinhuan Feng, Kwesi Oseitutu, and Clare-Marie Karat. 2003. Hands-Free, Speech-Based Navigation During Dictation: Difficulties, Consequences, and Solutions. Human-Computer Interaction 18 (2003), 229--257.Google ScholarDigital Library
- Ben Shneiderman. 2000. The Limits of Speech Recognition. Commun. ACM 43, 9 (Sept. 2000), 63--65. https://doi.org/10.1145/348941.348990Google ScholarDigital Library
- Ben Shneiderman, Catherine Plaisant, Maxine Cohen, Steven Jacobs, Niklas Elmqvist, and Nicholas Diakopoulos. 2016. Designing the User Interface: Strategies for Effective Human-Computer Interaction (6 ed.). Pearson, Harlow, UK. 616 pages.Google Scholar
- Hannu Soronen, Santtu Pakarinen, Mervi Hansen, Markku Turunen, Jaakko Hakulinen, Juho Hella, Juha-Pekka Rajaniemi, Aleksi Melto, and Tuuli Laivo. 2009. User Experience of Speech Controlled Media Center for Physically Disabled Users. In Proceedings of the 13th International MindTrek Conference: Everyday Life in the Ubiquitous Era (Tampere, Finland) (MindTrek '09). Association for Computing Machinery, New York, NY, USA, 2--5. https://doi.org/10.1145/1621841.1621843Google ScholarDigital Library
- Marilyn A Walker, Jeanne Fromer, Giuseppe Di Fabbrizio, Craig Mestel, and Don Hindle. 1998. What Can I Say? Evaluating a Spoken Language Interface to Email. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Los Angeles, CA, USA) (CHI '98). ACM Press/Addison-Wesley Publishing Co., USA, 582--589. https://doi.org/10.1145/274644.274722Google ScholarDigital Library
- Nicole Yankelovich. 1996. How Do Users Know What to Say? interactions 3, 6 (12 1996), 32--43. https://doi.org/10.1145/242485.242500Google Scholar
- Nicole Yankelovich, Gina-Anne Levow, and Matt Marx. 1995. Designing SpeechActs: Issues in Speech User Interfaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Denver, Colorado, USA) (CHI '95). ACM/Addison-Wesley Publishing Co., USA, 369--376. https://doi.org/10.1145/223904.223952Google ScholarDigital Library
Index Terms
- What Can I Say?: Effects of Discoverability in VUIs on Task Performance and User Experience
Recommendations
Learnability through Adaptive Discovery Tools in Voice User Interfaces
CHI EA '17: Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing SystemsThe invisible nature of VUIs has been attributed to challenging discoverability with VUIs. Low discoverability often leads to learnability issues. Researchers have designed visual tools for VUIs to help users learn as they go. However, few have used ...
System-initiated digressive proposals in automated human-computer telephone dialogues: the use of contrasting politeness strategies
System-initiated digressive proposals may be used to introduce new and unexpected information into automated telephone services. These digressions may be viewed as particularly pronounced forms of unsolicited interruptions as they contain information ...
NottReal: A Tool for Voice-based Wizard of Oz studies
CUI '20: Proceedings of the 2nd Conference on Conversational User InterfacesWe present NottReal, an application designed for simulating Voice User Interfaces (VUIs) in Wizard of Oz studies. We briefly discuss the premise and advantages of the Wizard of Oz method before moving onto introducing the design of the application, ...
Comments