Fraser Allison
ABSTRACT
Voice interaction is increasingly common in digital games, but it remains a notoriously difficult modality to design a satisfying experience for. This is partly due to limitations of speech recognition technology, and partly due to the inherent awkwardness we feel when performing some voice actions. We present a pattern language for voice interaction elements in games, to help game makers explore and describe common approaches to this design challenge. We define 25 design patterns, based on a survey of 449 videogames and 22 audiogames that use the player's voice as an input to affect the game state. The patterns express how games frame and structure voice input, and how voice input is used for selection, navigation, control and performance actions. Finally, we argue that academic research has been overly concentrated on a single one of these design patterns, due to an instrumental research focus and a lack of interest in the fictive dimension of videogames.
Supplemental Material
Available for Download
- Christopher Alexander, Sara Ishikawa, Murray Silverstein, Max Jacobson, Ingrid Fiksdahl-King, and Shlomo Angel. 1977. A Pattern Language: Towns, Buildings, Construction. Oxford University Press, New York.Google Scholar
- Fraser Allison, Marcus Carter, and Martin Gibbs. 2017. Word Play: A History of Voice Interaction in Digital Games. Games and Culture: 155541201774630.Google Scholar
- Valter Alves and Licinio Roque. 2010. A Pattern Language for Sound Design in Games. In Proceedings of the 5th Audio Mostly Conference: A Conference on Interaction with Sound (AM '10), 12:1--12:8. Google ScholarDigital Library
- Valter Alves and Licinio Roque. 2011. A Deck for Sound Design in Games: Enhancements Based on a Design Exercise. In Proceedings of the 8th International Conference on Advances in Computer Entertainment Technology (ACE '11), 34:1--34:8. Google ScholarDigital Library
- Peter Bayliss. 2007. Beings in the Game-world: Characters, Avatars, and Players. In Proceedings of the 4th Australasian Conference on Interactive Entertainment (IE '07), 4:1--4:6. Retrieved June 1, 2015 from http://dl.acm.org/citation.cfm?id=1367956.1367960 Google ScholarDigital Library
- BioWare. 2012. Mass Effect 3. Electronic Arts, Redwood City, CA.Google Scholar
- Staffan Björk. 2012. Gameplay Design Patterns for Public Games. EXPERIMEDIA.Google Scholar
- Staffan Björk and Jussi Holopainen. 2004. Patterns in Game Design. Charles River Media, Inc., Rockland, MA. Google ScholarDigital Library
- Staffan Björk, Sus Lundgren, and Jussi Holopainen. 2003. Game Design Patterns. In Proceedings of the 2003 DiGRA International Conference: Level Up (DiGRA '03).Google Scholar
- Staffan Björk and Johan Peitz. 2007. Understanding Pervasive Games through Gameplay Design Patterns. In DiGRA '07 - Proceedings of the 2007 DiGRA International Conference: Situated Play.Google Scholar
- Jenny Brusk and Staffan Björk. 2009. Gameplay Design Patterns for Game Dialogues. In Proceedings of DiGRA 2009.Google Scholar
- Capcom. 2005. Phoenix Wright: Ace Attorney. Capcom, Osaka, Japan.Google Scholar
- Marcus Carter, Fraser Allison, John Downs, and Martin Gibbs. 2015. Player Identity Dissonance and Voice Interaction in Games. In Proceedings of the 2015 Annual Symposium on Computer-Human Interaction in Play (CHI PLAY '15), 265--269. Google ScholarDigital Library
- Marcus Carter, John Downs, Bjorn Nansen, Mitchell Harrop, and Martin Gibbs. 2014. Paradigms of Games Research in HCI: A Review of 10 Years of Research at CHI. In Proceedings of the First ACM SIGCHI Annual Symposium on Computer-human Interaction in Play (CHI PLAY '14), 27--36. Google ScholarDigital Library
- Marcus Carter, Martin Gibbs, and Michael Arnold. 2012. Avatars, Characters, Players and Users: Multiple Identities at/in Play. In Proceedings of the 24th Australian Computer-Human Interaction Conference (OzCHI '12), 68--71. Google ScholarDigital Library
- Gabriel J. Cler, Talia Mittelman, Maia N. Braden, Geralyn Harvey Woodnorth, and Cara E. Stepp. 2017. Video Game Rehabilitation of Velopharyngeal Dysfunction: A Case Series. Journal of Speech, Language, and Hearing Research 60, 6S: 1800--1809.Google ScholarCross Ref
- Michael Cook, Mirjam Eladhari, Andy Nealen, Mike Treanor, Eddy Boxerman, Alex Jaffe, Paul Sottosanti, and Steve Swink. 2016. PCG-Based Game Design Patterns. arXiv:1610.03138 {cs}. Retrieved February 27, 2018 from http://arxiv.org/abs/1610.03138Google Scholar
- Chris Crawford. 2004. Chris Crawford on Interactive Storytelling. New Riders, Berkeley, CA. Google ScholarDigital Library
- Creative Assembly. 2014. Alien: Isolation. Sega, Tokyo.Google Scholar
- Crystal Dynamics. 2014. Tomb Raider: Definitive Edition. Square Enix, Tokyo.Google Scholar
- Augustin Dagoret, Jean-Louis Boudrand, Lorris Giovagnoli, Nicolas Terlon, Aymeric Thevenot, and Marie Vilain. 2016. Northern Lights. École Nationale du Jeu et des Médias Interactifs Numériques, Angoulême, France.Google Scholar
- Ola Davidsson, Johan Peitz, and Staffan Björk. 2004. Game Design Patterns for Mobile Games. Nokia Research Centre.Google Scholar
- Demerara Games. 2017. Resonance: The Lost Score. Demerara Games, Natal, Brazil.Google Scholar
- Claire Dormann, Jennifer R. Whitson, and Max Neuvians. 2013. Once More With Feeling: Game Design Patterns for Learning in the Affective Domain. Games and Culture 8, 4: 215--237.Google ScholarCross Ref
- Joris Dormans. 2013. Making Design Patterns Work. In Proceedings of the Second Workshop on Design Patterns in Games, DPG.Google Scholar
- Steven Dow, Manish Mehta, Ellie Harmon, Blair MacIntyre, and Michael Mateas. 2007. Presence and Engagement in an Interactive Drama. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '07), 1475--1484. Google ScholarDigital Library
- EA Canada. 2013. FIFA 14. EA Sports, Redwood City, CA.Google Scholar
- Thomas Erickson. 2000. Lingua Francas for Design: Sacred Places and Pattern Languages. In Proceedings of the 3rd Conference on Designing Interactive Systems: Processes, Practices, Methods, and Techniques (DIS '00), 357--368. Google ScholarDigital Library
- Gordon Fletcher and Ben Light. 2011. Interpreting digital gaming practices: SingStar as a technology of work. Retrieved March 30, 2015 from http://usir.salford.ac.uk/17245/Google Scholar
- Freedom Crow. 2017. Yasuhati. Freem Inc., Nishinomiya, Japan.Google Scholar
- Erich Gamma, Richard Helm, Ralph Johnson, and John Vlissides. 1995. Design Patterns: Elements of Reusable Object-oriented Software. Addison-Wesley Longman Publishing Co., Inc., Boston, MA, USA. Google ScholarDigital Library
- Peter Gorniak and Deb Roy. 2005. Probabilistic Grounding of Situated Speech Using Plan Recognition and Reference Resolution. In Proceedings of the 7th International Conference on Multimodal Interfaces (ICMI '05), 138--143. Google ScholarDigital Library
- Susumu Harada, Jacob O. Wobbrock, and James A. Landay. 2011. Voice Games: Investigation into the Use of Non-speech Voice Input for Making Computer Games More Accessible. In Proceedings of the 13th IFIP TC 13 International Conference on Human-computer Interaction - Volume Part I (INTERACT'11), 11--29. Google ScholarDigital Library
- Harmonix. 2004. Karaoke Revolution Volume 2. Konami, Tokyo.Google Scholar
- Kenneth Hullett and Jim Whitehead. 2010. Design Patterns in FPS Levels. In Proceedings of the Fifth International Conference on the Foundations of Digital Games (FDG '10), 78--85. Google ScholarDigital Library
- Takeo Igarashi and John F. Hughes. 2001. Voice As Sound: Using Non-verbal Voice Input for Interactive Control. In Proceedings of the 14th Annual ACM Symposium on User Interface Software and Technology (UIST '01), 155--156. Google ScholarDigital Library
- Indomitus Games. 2015. In Verbis Virtus. Indomitus Games, Borgo a Mozzano, Italy.Google Scholar
- Iridium Studios. 2015. There Came an Echo. Iridium Studios, Los Angeles.Google Scholar
- W. Lewis Johnson and Andre Valente. 2009. Tactical Language and Culture Training Systems: Using AI to Teach Foreign Languages and Cultures. AI Magazine 30, 2: 72.Google ScholarDigital Library
- Joyce Hakansson Associates. 1983. I'm Hiding. Milton Bradley, East Longmeadow, MA.Google Scholar
- Peter H. Kahn, Nathan G. Freier, Takayuki Kanda, Hiroshi Ishiguro, Jolina H. Ruckert, Rachel L. Severson, and Shaun K. Kane. 2008. Design Patterns for Sociality in Human-Robot Interaction. In Proceedings of the 3rd ACM/IEEE International Conference on Human Robot Interaction (HRI '08), 97--104. Google ScholarDigital Library
- Anuj Kumar, Pooja Reddy, Anuj Tewari, Rajat Agrawal, and Matthew Kam. 2012. Improving Literacy in Developing Countries Using Speech Recognition-supported Games on Mobile Devices. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '12), 1149--1158. Google ScholarDigital Library
- Labgoo. 2011. Pah! Labgoo, Tel Aviv.Google Scholar
- Petri Lankoski and Staffan Björk. 2007. Gameplay Design Patterns for Believable Non-Player Characters. In Proceedings of the 2007 DiGRA International Conference: Situated Play (DiGRA '07).Google Scholar
- Petri Lankoski, Anja Johansson, Benny Karlsson, Staffan Björk, and Pierangelo Dell'Acqua. 2011. AI Design for Believable Characters via Gameplay Design Patterns. In Business, Technological and Social Dimensions of Computer Games: Multidisciplinary Developments. IGI Global, 15.Google Scholar
- Chris Lewis, Noah Wardrip-Fruin, and Jim Whitehead. 2012. Motivational Game Design Patterns of 'Ville Games. In Proceedings of the International Conference on the Foundations of Digital Games (FDG '12), 172--179. Google ScholarDigital Library
- Looking West Apps. 2013. ATC Voice. Looking West Apps, Denver, CO.Google Scholar
- Marta Lopes, João Magalhães, and Sofia Cavaco. 2016. A Voice-controlled Serious Game for the Sustained Vowel Exercise. In Proceedings of the 13th International Conference on Advances in Computer Entertainment Technology (ACE '16), 32:1--32:6. Google ScholarDigital Library
- Ian McGraw, On Yoshimoto, and Stephanie Seneff. 2009. Speech-enabled Card Games for Incidental Vocabulary Acquisition in a Foreign Language. Speech Communication 51, 10: 1006--1023. Google ScholarDigital Library
- Bria Mears and Jichen Zhu. 2017. Design Patterns for Silent Player Characters in Narrative-driven Games. In Proceedings of the 12th International Conference on the Foundations of Digital Games (FDG '17), 59:1--59:4. Google ScholarDigital Library
- David Milam, Lyn Bartram, and Magy Seif El-Nasr. 2012. Design Patterns of Focused Attention. In Proceedings of the First Workshop on Design Patterns in Games (DPG '12), 5:1--5:8. Google ScholarDigital Library
- David Milam and Magy Seif El-Nasr. 2010. Design Patterns to Guide Player Movement in 3D Games. In Proceedings of the 5th ACM SIGGRAPH Symposium on Video Games (Sandbox '10), 37--42. Google ScholarDigital Library
- Beckett Mufson. 2017. Watch People Play a Hilarious, Frustrating, Voice-Controlled iPhone Game. VICE Creators. Retrieved April 14, 2018 from https://creators.vice.com/en_au/article/vvjvk9/hilarious-frustrating-voice-controlled-iphone-gameGoogle Scholar
- Andrés Adolfo Navarro-Newball, Diego Loaiza, Claudia Oviedo, Andrés Darío Castillo, A. Portilla, Diego Linares, and Gloria Inés Álvarez. 2014. Talking to Teo: Video game supported speech therapy. Entertainment Computing 5, 4: 401--412.Google ScholarCross Ref
- Neversoft. 2008. Guitar Hero: World Tour. Activision, Santa Monica, CA.Google Scholar
- Nintendo Entertainment Analysis & Development. 2005. Nintendogs. Nintendo, Kyoto, Japan.Google Scholar
- Nintendo Entertainment Analysis & Development. 2007. The Legend of Zelda: Phantom Hourglass. Nintendo, Kyoto, Japan.Google Scholar
- Yue Pan and Erik Stolterman. 2013. Pattern Language and HCI: Expectations and Experiences. In CHI '13 Extended Abstracts on Human Factors in Computing Systems (CHI EA '13), 1989--1998. Google ScholarDigital Library
- James R. Parker and John Heerema. 2008. Audio Interaction in Computer Mediated Games. International Journal of Computer Games Technology 2008: 1:1--1:8. Google ScholarDigital Library
- Playground Games. 2014. Forza Horizon 2. Microsoft Studios, Redmond, WA.Google Scholar
- RDI Video Systems. 1984. Thayer's Quest. RDI Video Systems, Carlsbad, CA.Google Scholar
- Gabriel Rivera, Kenneth Hullett, and Jim Whitehead. 2012. Enemy NPC Design Patterns in Shooter Games. In Proceedings of the First Workshop on Design Patterns in Games (DPG '12), 6:1--6:8. Google ScholarDigital Library
- Zak Rubin and Sri Kurniawan. 2013. Speech Adventure: Using Speech Recognition for Cleft Speech Therapy. In Proceedings of the 6th International Conference on Pervasive Technologies Related to Assistive Environments (PETRA '13), 35:1--35:4. Google ScholarDigital Library
- Serdar Sali, Noah Wardrip-Fruin, Steven Dow, Michael Mateas, Sri Kurniawan, Aaron A. Reed, and Ronald Liu. 2010. Playing with Words: From Intuition to Evaluation of Game Dialogue Interfaces. In Proceedings of the Fifth International Conference on the Foundations of Digital Games (FDG '10), 179--186. Google ScholarDigital Library
- Katarina Segerståhl and Timo Jokela. 2006. Usability of Interaction Patterns. In CHI '06 Extended Abstracts on Human Factors in Computing Systems (CHI EA '06), 1301--1306. Google ScholarDigital Library
- Ben Shneiderman. 2000. The Limits of Speech Recognition. Communications of the ACM 43, 63--65. Google ScholarDigital Library
- Gillian Smith, Ryan Anderson, Brian Kopleck, Zach Lindblad, Lauren Scott, Adam Wardell, Jim Whitehead, and Michael Mateas. 2011. Situating Quests: Design Patterns for Quest and Level Design in Role-Playing Games. In Interactive Storytelling (Lecture Notes in Computer Science), 326--329. Google ScholarDigital Library
- Adam J. Sporka, Sri H. Kurniawan, Murni Mahmud, and Pavel Slavík. 2006. Non-speech Input and Speech Recognition for Real-time Control of Computer Games. 213--220. Google ScholarDigital Library
- Chek Tien Tan, Andrew Johnston, Kirrie Ballard, Samuel Ferguson, and Dharani Perera-Schulz. 2013. sPeAK-MAN: Towards Popular Gameplay for Speech Therapy. In Proceedings of The 9th Australasian Conference on Interactive Entertainment: Matters of Life and Death (IE '13), 28:1--28:4. Google ScholarDigital Library
- Zachary O. Toups, William A. Hamilton, and Sultan A. Alharthi. 2016. Playing at Planning: Game Design Patterns from Disaster Response Practice. In Proceedings of the 2016 Annual Symposium on Computer-Human Interaction in Play (CHI PLAY '16), 362--375. Google ScholarDigital Library
- Mike Treanor, Alexander Zook, Mirjam P Eladhari, Julian Togelius, Gillian Smith, Michael Cook, Tommy Thompson, Brian Magerko, John Levine, and Adam Smith. 2015. AI-based game design patterns. In Proceedings of the 10th International Conference on the Foundations of Digital Games 2015 (FDG 2015), 8. Retrieved from http://www.fdg2015.org/papers/fdg2015_paper_23.pdfGoogle Scholar
- Paul Turowski and Simon Hutchinson. 2017. Plurality Spring.Google Scholar
- Ubisoft Montreal. 2006. Rainbow Six: Vegas. Ubisoft, Paris.Google Scholar
- Ubisoft Shanghai. 2008. Tom Clancy's EndWar. Ubisoft, Paris.Google Scholar
- Eduardo Velloso and Marcus Carter. 2016. The Emergence of EyePlay: A Survey of Eye Interaction in Games. In Proceedings of the 2016 Annual Symposium on Computer-Human Interaction in Play (CHI PLAY '16), 171--185. Google ScholarDigital Library
- Marco Filipe Ganança Vieira, Hao Fu, Chong Hu, Nayoung Kim, and Sudhanshu Aggarwal. 2014. PowerFall: A Voice-controlled Collaborative Game. In Proceedings of the First ACM SIGCHI Annual Symposium on Computer-human Interaction in Play (CHI PLAY '14), 395--398. Google ScholarDigital Library
- Vivarium. 1999. Seaman. Sega, Tokyo.Google Scholar
- Greg Wadley, Marcus Carter, and Martin Gibbs. 2015. Voice in Virtual Worlds: The Design, Use, and Influence of Voice Chat in Online Play. Human--Computer Interaction 30, 3--4: 336--365. Google ScholarDigital Library
- Greg Wadley and Nicolas Ducheneaut. 2009. The 'Out-of-Avatar Experience': Object-focused Collaboration in Second Life. In Proceedings of the 11th European Conference on Computer Supported Cooperative Work (ECSCW 2009), 323--342.Google ScholarCross Ref
- José P. Zagal, Staffan Björk, and Chris Lewis. 2013. Dark Patterns in the Design of Games. In Proceedings of the 8th International Conference on the Foundations of Digital Games (FDG 2013), 39--46.Google Scholar
Index Terms
- Design Patterns for Voice Interaction in Games
Recommendations
Frame Analysis of Voice Interaction Gameplay
CHI '19: Proceedings of the 2019 CHI Conference on Human Factors in Computing SystemsVoice control is an increasingly common feature of digital games, but the experience of playing with voice control is often hampered by feelings of embarrassment and dissonance. Past research has recognised these tensions, but has not offered a general ...
Enemy NPC design patterns in shooter games
DPG '12: Proceedings of the First Workshop on Design Patterns in GamesGameplay in single-player shooter games primarily consists of combat with Non-player Characters (NPCs). However, there has been little research done to study how enemy NPCs affect player behavior in shooter games. As a consequence, there is no formal ...
Exploring aesthetical gameplay design patterns: camaraderie in four games
MindTrek '10: Proceedings of the 14th International Academic MindTrek Conference: Envisioning Future Media EnvironmentsThis paper explores how a vocabulary supporting design-related discussions of gameplay preferences can be developed. Using the preference of experiencing camaraderie as an example, we have analyzed four games: the board games Space Alert and Battlestar ...
Comments