Matthew Sidji
ABSTRACT
Games that feature multiple players, limited communication, and partial information are particularly challenging for AI agents. In the cooperative card game Hanabi, which possesses all of these attributes, AI agents fail to achieve scores comparable to even first-time human players. Through an observational study of three mixed-skill Hanabi play groups, we identify the techniques used by humans that help to explain their superior performance compared to AI. These concern physical artefact manipulation, coordination play, role establishment, and continual rule negotiation. Our findings extend previous accounts of human performance in Hanabi, which are purely in terms of theory-of-mind reasoning, by revealing more precisely how this form of collective decision-making is enacted in skilled human play. Our interpretation points to a gap in the current capabilities of AI agents to perform cooperative tasks.
Supplemental Material
- A. Adadi and M. Berrada. 2018. Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI). IEEE Access 6(2018), 38–60. https://doi.org/10.1109/ACCESS.2018.2870052Google Scholar
Cross Ref
- Andrew Anderson, Jonathan Dodge, Amrita Sadarangani, Zoe Juozapaitis, Evan Newman, Jed Irvine, Souti Chattopadhyay, Matthew Olson, Alan Fern, and Margaret Burnett. 2020. Mental Models of Mere Mortals with Explanations of Reinforcement Learning. ACM Transactions on Interactive Intelligent Systems 10 (2020), 1–37. https://doi.org/10.1145/3366485Google ScholarDigital Library
- Gagan Bansal, Besmira Nushi, Ece Kamar, Walter S. Lasecki, Daniel S. Weld, and Eric Horvitz. 2019. Beyond Accuracy: The Role of Mental Models in Human-AI Team Performance. Proceedings of the AAAI Conference on Human Computation and Crowdsourcing 7, 1 (Oct. 2019), 2–11. https://ojs.aaai.org/index.php/HCOMP/article/view/5285 Section: Technical Papers.Google ScholarCross Ref
- Shaira Baptista, Greg Wadley, Dominique Bird, Brian Oldenburg, and Jane Speight. 2020. Acceptability of an Embodied Conversational Agent for Type 2 Diabetes Self-Management Education and Support via a Smartphone App: Mixed Methods Study. JMIR Mhealth Uhealth 8, 7 (July 2020), 1–13. https://doi.org/10.2196/17038Google ScholarCross Ref
- Nolan Bard, Jakob N. Foerster, Sarath Chandar, Neil Burch, Marc Lanctot, H. Francis Song, Emilio Parisotto, Vincent Dumoulin, Subhodeep Moitra, Edward Hughes, Iain Dunning, Shibl Mourad, Hugo Larochelle, Marc G. Bellemare, and Michael Bowling. 2020. The Hanabi challenge: A new frontier for AI research. Artificial Intelligence 280 (March 2020), 103216. https://doi.org/10.1016/j.artint.2019.103216Google ScholarDigital Library
- Jonathan Bassen, Bharathan Balaji, Michael Schaarschmidt, Candace Thille, Jay Painter, Dawn Zimmaro, Alex Games, Ethan Fast, and John C. Mitchell. 2020. Reinforcement Learning for the Adaptive Scheduling of Educational Activities. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems(CHI ’20). Association for Computing Machinery, New York, NY, USA, 1–12. https://doi.org/10.1145/3313831.3376518 event-place: Honolulu, HI, USA.Google ScholarDigital Library
- Cindy Beaudoin, Élizabel Leblanc, Charlotte Gagner, and Miriam H. Beauchamp. 2020. Systematic Review and Inventory of Theory of Mind Measures for Young Children. Frontiers in Psychology 10 (2020), 1–4. https://www.frontiersin.org/articles/10.3389/fpsyg.2019.02905Google ScholarCross Ref
- Rhyse Bendell, Jessica Williams, Stephen M. Fiore, and Florian Jentsch. 2021. Towards Artificial Social Intelligence: Inherent Features, Individual Differences, Mental Models, and Theory of Mind. In Advances in Neuroergonomics and Cognitive Engineering, Hasan Ayaz, Umer Asgher, and Lucas Paletta (Eds.). Springer International Publishing, Cham, 20–28. https://doi.org/10.1007/978-3-030-80285-1_3Google ScholarCross Ref
- Kenneth L. Bettenhausen. 1991. Five Years of Groups Research: What We Have Learned and What Needs to Be Addressed. Journal of Management 17, 2 (June 1991), 345–381. https://doi.org/10.1177/014920639101700205Google ScholarCross Ref
- Francesco Biondi, Ignacio Alvarez, and Kyeong-Ah Jeong. 2019. Human–Vehicle Cooperation in Automated Driving: A Multidisciplinary Review and Appraisal. International Journal of Human–Computer Interaction 35, 11 (July 2019), 932–946. https://doi.org/10.1080/10447318.2018.1561792Google ScholarCross Ref
- Virginia Braun and Victoria Clarke. 2012. Thematic analysis.In APA handbook of research methods in psychology, Vol 2: Research designs: Quantitative, qualitative, neuropsychological, and biological.American Psychological Association, Washington, DC, US, 57–71. https://doi.org/10.1037/13620-004Google ScholarCross Ref
- Noam Brown and Tuomas Sandholm. 2019. Superhuman AI for multiplayer poker. Science 365, 6456 (2019), 885–890. Publisher: American Association for the Advancement of Science.Google Scholar
- Lucian Buşoniu, Robert Babuška, and Bart De Schutter. 2010. Multi-agent Reinforcement Learning: An Overview. In Innovations in Multi-Agent Systems and Applications - 1, Dipti Srinivasan and Lakhmi C. Jain (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 183–221. https://doi.org/10.1007/978-3-642-14435-6_7Google ScholarCross Ref
- Janis A. Cannon-Bowers, Eduardo Salas, and Sharolyn Converse. 1993. Shared mental models in expert team decision making.In Individual and group decision making: Current issues.Lawrence Erlbaum Associates, Inc, Hillsdale, NJ, US, 221–246.Google Scholar
- John M. Carroll. 2003. CHAPTER 1 - Introduction: Toward a Multidisciplinary Science of Human-Computer Interaction. In HCI Models, Theories, and Frameworks, John M. Carroll (Ed.). Morgan Kaufmann, San Francisco, 1–9. https://doi.org/10.1016/B978-155860808-5/50001-0Google ScholarCross Ref
- Rohan Chandra, Aniket Bera, and Dinesh Manocha. 2020. StylePredict: Machine Theory of Mind for Human Driver Behavior From Trajectories. CoRR abs/2011.04816(2020), 1–12. https://arxiv.org/abs/2011.04816 arXiv:2011.04816.Google Scholar
- Gifford Cheung. 2013. Card Board: A Flexible Environment for Any Game, Anyone, Any Moment. In CHI ’13 Extended Abstracts on Human Factors in Computing Systems (Paris, France) (CHI EA ’13). Association for Computing Machinery, New York, NY, USA, 2627–2630. https://doi.org/10.1145/2468356.2479480Google ScholarDigital Library
- Mark Coeckelbergh. 2016. Responsibility and the Moral Phenomenology of Using Self-Driving Cars. Applied Artificial Intelligence 30, 8 (Sept. 2016), 748–757. https://doi.org/10.1080/08839514.2016.1229759Google ScholarCross Ref
- Philip R. Cohen and Hector J. Levesque. 1991. Teamwork. Noûs 25, 4 (1991), 487–512. https://doi.org/10.2307/2216075Google ScholarCross Ref
- Nancy J. Cooke, Eduardo Salas, Janis A. Cannon-Bowers, and Renée J. Stout. 2000. Measuring Team Knowledge. Human Factors 42, 1 (2000), 151–173. https://doi.org/10.1518/001872000779656561Google ScholarCross Ref
- Christopher Cox, Jessica De Silva, Philip Deorsey, Franklin H. J. Kenter, Troy Retter, and Josh Tobin. 2015. How to Make the Perfect Fireworks Display: Two Strategies for Hanabi. Mathematics Magazine 88, 5 (Dec. 2015), 323–336. https://doi.org/10.4169/math.mag.88.5.323Google ScholarCross Ref
- Allan Dafoe, Edward Hughes, Yoram Bachrach, Tantum Collins, Kevin R. McKee, Joel Z. Leibo, Kate Larson, and Thore Graepel. 2020. Open Problems in Cooperative AI. CoRR abs/2012.08630(2020), 1–35. https://arxiv.org/abs/2012.08630 arXiv:2012.08630.Google Scholar
- Allan Dafoe, Edward Hughes, Yoram Bachrach, Tantum Collins, Kevin R. McKee, Joel Z. Leibo, Kate Larson, and Thore Graepel. 2020. Open Problems in Cooperative AI. CoRR abs/2012.08630(2020), 1–13. https://arxiv.org/abs/2012.08630 arXiv:2012.08630.Google Scholar
- Alan Dix. 1997. Challenges for Cooperative Work on the Web: An Analytical Approach. In Groupware and the World Wide Web, Richard Bentley, Uwe Busbach, David Kerr, and Klaas Sikkel (Eds.). Springer Netherlands, Dordrecht, 25–46. https://doi.org/10.1007/978-94-011-5756-8_2Google ScholarCross Ref
- Elian Fink, Sander Begeer, Candida C Peterson, Virginia Slaughter, and Marc de Rosnay. 2015. Friends, friendlessness, and the social consequences of gaining a theory of mind. British Journal of Developmental Psychology 33, 1 (March 2015), 27–30. https://doi.org/10.1111/bjdp.12080Google ScholarCross Ref
- Nick Flor and Edwin Hutchins. 1991. Analyzing distributed cognition in software teams: A case study of team programming during perfective software maintenance. Empirical studies of programmers 4, 3 (Jan. 1991), 36.Google Scholar
- Jakob Foerster, Ioannis Alexandros Assael, Nando de Freitas, and Shimon Whiteson. 2016. Learning to Communicate with Deep Multi-Agent Reinforcement Learning. In Advances in Neural Information Processing Systems, D. Lee, M. Sugiyama, U. Luxburg, I. Guyon, and R. Garnett (Eds.). Vol. 29. Curran Associates, Inc., Red Hook NY, United States, 1–9. https://proceedings.neurips.cc/paper/2016/file/c7635bfd99248a2cdef8249ef7bfbef4-Paper.pdfGoogle Scholar
- Chris Frith and Uta Frith. 2005. Theory of mind. Current biology 15, 17 (2005), R644–R645. https://doi.org/10.1016/j.cub.2005.08.041 Publisher: Elsevier.Google ScholarCross Ref
- Berserk Games. 2015. Tabletop Simulator. [CD-ROM].Google Scholar
- Katy Ilonka Gero, Zahra Ashktorab, Casey Dugan, Qian Pan, James Johnson, Werner Geyer, Maria Ruiz, Sarah Miller, David R. Millen, Murray Campbell, Sadhana Kumaravel, and Wei Zhang. 2020. Mental Models of AI Agents in a Cooperative Game Setting. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. Association for Computing Machinery, Honolulu, Hawaii, USA, 1–12. https://doi.org/10.1145/3313831.3376316Google ScholarDigital Library
- Noah Goodall. 12. Ethical Decision Making During Automated Vehicle Crashes. Transportation Research Record: Journal of the Transportation Research Board 2424 (Jan. 12), 58–65. https://doi.org/10.3141/2424-07Google ScholarCross Ref
- Eva Tallula Gottwald, Markus Eger, and Chris Martens. 2018. I See What You See: Integrating Eye Tracking into Hanabi Playing Agents.. In I See What You See: Integrating Eye Tracking into Hanabi Playing Agents.AIIDE 2018, Edmonton, Alberta, Canada, 1–13.Google Scholar
- Victoria Groom and Clifford Nass. 2007. Can robots be teammates?: Benchmarks in human–robot teams. Interaction Studies 8, 3 (2007), 483–500. https://doi.org/10.1075/is.8.3.10groGoogle ScholarCross Ref
- Rotem D Guttman, Jessica Hammer, Erik Harpstead, and Carol J Smith. 2021. Play for Real (ism)-Using Games to Predict Human-AI interactions in the Real World. Proceedings of the ACM on Human-Computer Interaction 5, CHI PLAY(2021), 1–17. Publisher: ACM New York, NY, USA.Google Scholar
- Matthew Guzdial, Nicholas Liao, Jonathan Chen, Shao-Yu Chen, Shukan Shah, Vishwa Shah, Joshua Reno, Gillian Smith, and Mark O. Riedl. 2019. Friend, Collaborator, Student, Manager: How Design of an AI-Driven Game Level Editor Affects Creators. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems(CHI ’19). Association for Computing Machinery, New York, NY, USA, 1–13. https://doi.org/10.1145/3290605.3300854 event-place: Glasgow, Scotland Uk.Google ScholarDigital Library
- J. Richard Hackman. 1978. The design of work in the 1980s. Organizational Dynamics 7, 1 (1978), 3–17. https://doi.org/10.1016/0090-2616(78)90031-1Google ScholarCross Ref
- Hengyuan Hu, Adam Lerer, Alex Peysakhovich, and Jakob Foerster. 2020. “Other-Play” for Zero-Shot Coordination. In “Other-Play” for Zero-Shot Coordination, Daumé Hal, III and Singh Aarti (Eds.). Vol. 119. PMLR, Proceedings of Machine Learning Research, 4399–4410. https://proceedings.mlr.press/v119/hu20a.htmlGoogle Scholar
- Edwin Hutchins. 1995. Cognition in the wild.The MIT Press, Cambridge, MA, US. Pages: xviii, 381.Google Scholar
- Edwin Hutchins and Tove Klausen. 1996. Distributed cognition in an airline cockpit. Cambridge University Press, Cambridge, 15–34. https://doi.org/10.1017/CBO9781139174077.002Google ScholarCross Ref
- Angel Hsing-Chi Hwang and Andrea Stevenson Won. 2021. IdeaBot: Investigating Social Facilitation in Human-Machine Team Creativity. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. ACM, Yokohama, Japan, Article 712. https://doi.org/10.1145/3411764.3445270Google ScholarDigital Library
- Deborah G. Johnson and Mario Verdicchio. 2019. AI, agency and responsibility: the VW fraud case and beyond. Ai & Society 34, 3 (Sept. 2019), 639–647. https://doi.org/10.1007/s00146-017-0781-9Google ScholarDigital Library
- Garry Kasparov. 2018. Chess, a Drosophila of reasoning. Science 362, 6419 (2018), 1087–1087. https://doi.org/10.1126/science.aaw2221Google ScholarCross Ref
- Lindsay Larson and Leslie A. DeChurch. 2020. Leading teams in the digital age: Four perspectives on technology and what they mean for leading teams. The Leadership Quarterly 31, 1 (Feb. 2020), 1–13. https://doi.org/10.1016/j.leaqua.2019.101377Google ScholarCross Ref
- Reeva Lederman and Robert B Johnston. 2011. Decision support or support for situated choice: lessons for system design from effective manual systems. European Journal of Information Systems 20, 5 (Sept. 2011), 510–528. https://doi.org/10.1057/ejis.2011.11Google ScholarCross Ref
- Adam Lerer, Hengyuan Hu, Jakob Foerster, and Noam Brown. 2020. Improving Policies via Search in Cooperative Partially Observable Games. Proceedings of the AAAI Conference on Artificial Intelligence 34, 05 (April 2020), 7187–7194. https://doi.org/10.1609/aaai.v34i05.6208Google ScholarCross Ref
- Ariel Levy, Monica Agrawal, Arvind Satyanarayan, and David Sontag. 2021. Assessing the Impact of Automated Suggestions on Decision Making: Domain Experts Mediate Model Errors but Take Less Initiative. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. ACM, Yokohama, Japan, 1–13. https://doi.org/10.1145/3411764.3445522Google ScholarDigital Library
- Claire Liang, Julia Proft, Erik Andersen, and Ross A. Knepper. 2019. Implicit Communication of Actionable Information in Human-AI Teams. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems(CHI ’19). Association for Computing Machinery, New York, NY, USA, 1–13. https://doi.org/10.1145/3290605.3300325 event-place: Glasgow, Scotland Uk.Google ScholarDigital Library
- M. Eger, C. Martens, and M. A. Cordoba. 2017. An intentional AI for Hanabi. In 2017 IEEE Conference on Computational Intelligence and Games (CIG). IEEE, New York, NY, USA, 68–75. https://doi.org/10.1109/CIG.2017.8080417 Journal Abbreviation: 2017 IEEE Conference on Computational Intelligence and Games (CIG).Google ScholarDigital Library
- M. Eger, C. Martens, P. S. Chacón, M. A. Córdoba, and J. Hidalgo-Cespedes. 2021. Operationalizing Intentionality to Play Hanabi With Human Players. IEEE Transactions on Games 13, 4 (Dec. 2021), 388–397. https://doi.org/10.1109/TG.2020.3009359Google ScholarCross Ref
- Gregor McEwan and Carl Gutwin. 2016. Chess as a Conversation: Artefact-Based Communication in Online Competitive Board Games. In Proceedings of the 19th International Conference on Supporting Group Work(GROUP ’16). Association for Computing Machinery, New York, NY, USA, 21–30. https://doi.org/10.1145/2957276.2957314 event-place: Sanibel Island, Florida, USA.Google ScholarDigital Library
- B Meijering, Hedderik Rijn, Niels Taatgen, and Rineke Verbrugge. 2011. I do know what you think I think: Second-order theory of mind in strategic games is not that difficult. Cogsci - Cognitive Science Society 33, 2 (Jan. 2011), 2486–2491.Google Scholar
- Tim Merritt and Kevin McGee. 2012. Protecting Artificial Team-Mates: More Seems like Less. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems(CHI ’12). Association for Computing Machinery, New York, NY, USA, 2793–2802. https://doi.org/10.1145/2207676.2208680Google ScholarDigital Library
- Susan Mohammed and Brad C. Dumville. 2001. Team mental models in a team knowledge framework: expanding theory and measurement across disciplinary boundaries. Journal of Organizational Behavior 22, 2 (2001), 89–106. https://doi.org/10.1002/job.86Google ScholarCross Ref
- OpenAI, :, Christopher Berner, Greg Brockman, Brooke Chan, Vicki Cheung, Przemysław Dębiak, Christy Dennison, David Farhi, Quirin Fischer, Shariq Hashme, Chris Hesse, Rafal Józefowicz, Scott Gray, Catherine Olsson, Jakub Pachocki, Michael Petrov, Henrique P. d. O. Pinto, Jonathan Raiman, Tim Salimans, Jeremy Schlatter, Jonas Schneider, Szymon Sidor, Ilya Sutskever, Jie Tang, Filip Wolski, and Susan Zhang. 2019. Dota 2 with Large Scale Deep Reinforcement Learning. https://doi.org/10.48550/ARXIV.1912.06680Google ScholarCross Ref
- Judith Orasanu. 1990. Shared mental models and crew decision making. In Proceedings from 1th Annual Conference. C.S.S. Pod. Psychology Press, New York, USA, 1066–1066.Google Scholar
- Hirotaka Osawa. 2015. Solving hanabi: Estimating hands by opponent’s actions in cooperative game with incomplete information. In Computer Poker and Imperfect Information - Papers Presented at the 29th AAAI Conference on Artificial Intelligence, Technical Report(AAAI Workshop - Technical Report). AI Access Foundation, United States, 37–43. Publisher Copyright: Copyright © 2015, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved.; 29th AAAI Conference on Artificial Intelligence, AAAI 2015 ; Conference date: 25-01-2015 Through 30-01-2015.Google Scholar
- Jon Peoble. 2013. How good is your score really? The statistics. https://boardgamegeek.com/thread/1005947/how-good-your-score-really-statisticsGoogle Scholar
- Leon Reicherts, Yvonne Rogers, Licia Capra, Ethan Wood, Tu Dinh Duong, and Neil Sebire. 2022. It’s Good to Talk: A Comparison of Using Voice Versus Screen-Based Interactions for Agent-Assisted Tasks. ACM Transactions on Computer-Human Interaction 29, 3 (Jan. 2022), 1–41. https://doi.org/10.1145/3484221Google ScholarDigital Library
- Evan F. Risko and Sam J. Gilbert. 2016. Cognitive Offloading. Trends in Cognitive Sciences 20, 9 (2016), 676–688. https://doi.org/10.1016/j.tics.2016.07.002Google ScholarCross Ref
- Jennifer Rix. 2022. From Tools to Teammates: Conceptualizing Humans’ Perception of Machines as Teammates with a Systematic Literature Review. In Proceedings of the 55th Hawaii International Conference on System Sciences. IEEE, online, 1–10. https://doi.org/10.24251/HICSS.2022.048Google ScholarCross Ref
- Melissa J. Rogerson, Martin R. Gibbs, and Wally Smith. 2017. What Can We Learn from Eye Tracking Boardgame Play?. In Extended Abstracts Publication of the Annual Symposium on Computer-Human Interaction in Play(CHI PLAY ’17 Extended Abstracts). Association for Computing Machinery, New York, NY, USA, 519–526. https://doi.org/10.1145/3130859.3131314 event-place: Amsterdam, The Netherlands.Google ScholarDigital Library
- Melissa J. Rogerson, Martin R. Gibbs, and Wally Smith. 2018. Cooperating to Compete: The Mutuality of Cooperation and Competition in Boardgame Play. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems(CHI ’18). Association for Computing Machinery, New York, NY, USA, 1–13. https://doi.org/10.1145/3173574.3173767Google ScholarDigital Library
- W.B. Rouse, J.A. Cannon-Bowers, and E. Salas. 1992. The role of mental models in team performance in complex systems. IEEE Transactions on Systems, Man, and Cybernetics 22, 6(1992), 1296–1308. https://doi.org/10.1109/21.199457Google ScholarCross Ref
- Aron Sarmasi, Timothy Zhang, Chu-Hung Cheng, Huyen Pham, Xuanchen Zhou, Duong Nguyen, Soumil Shekdar, and Joshua McCoy. 2021. HOAD: The Hanabi Open Agent Dataset. In Proceedings of the 20th International Conference on Autonomous Agents and MultiAgent Systems(AAMAS ’21). International Foundation for Autonomous Agents and Multiagent Systems, Richland, SC, 1646–1648.Google ScholarDigital Library
- Eisuke Sato and Hirotaka Osawa. 2020. Reducing Partner’s Cognitive Load by Estimating the Level of Understanding in the Cooperative Game Hanabi. In Advances in Computer Games, Tristan Cazenave, Jaap van den Herik, Abdallah Saffidine, and I-Chen Wu (Eds.). Springer International Publishing, Cham, 11–23.Google Scholar
- David Silver, Julian Schrittwieser, Karen Simonyan, Ioannis Antonoglou, Aja Huang, Arthur Guez, Thomas Hubert, Lucas Baker, Matthew Lai, Adrian Bolton, Yutian Chen, Timothy Lillicrap, Fan Hui, Laurent Sifre, George van den Driessche, Thore Graepel, and Demis Hassabis. 2017. Mastering the game of Go without human knowledge. Nature 550, 7676 (Oct. 2017), 354–359. https://doi.org/10.1038/nature24270Google ScholarCross Ref
- Stephen Sniderman. 1999. Unwritten rules. The Life of Games 1, 1 (1999), 2–7.Google Scholar
- Renée J. Stout, Janis A. Cannon-Bowers, Eduardo Salas, and Dana M. Milanovich. 1999. Planning, Shared Mental Models, and Coordinated Performance: An Empirical Link Is Established. Human Factors 41, 1 (1999), 61–71. https://doi.org/10.1518/001872099779577273Google ScholarCross Ref
- Lucy Suchman. 2006. Human-machine reconfigurations: plans and situated actions. Cambridge University Press, Cambridge. https://cds.cern.ch/record/1991545Google Scholar
- Gareth Terry, Nikki Hayfield, Victoria Clarke, and Virginia Braun. 2017. Thematic analysis. The SAGE handbook of qualitative research in psychology 2 (2017), 17–37.Google Scholar
- Phil Turner. 2016. Post-cognitive Interaction. Springer International Publishing, online.Google Scholar
- Kyriakos G. Vamvoudakis and Frank L. Lewis. 2011. Multi-player non-zero-sum games: Online adaptive learning solution of coupled Hamilton–Jacobi equations. Automatica 47, 8 (2011), 1556–1569. https://doi.org/10.1016/j.automatica.2011.03.005Google ScholarDigital Library
- Karel Van Den Bosch and Adelbert Bronkhorst. 2018. Human-AI cooperation to benefit military decision making. In Proceedings of the NATO IST-160 Specialist’ meeting on Big Data and Artificial Intelligence for Military Decision Making. NATO, Bordeaux, France, 48 – 61.Google Scholar
- Rineke Verbrugge and Lisette Mol. 2008. Learning to Apply Theory of Mind. Journal of Logic, Language and Information 17, 4 (Oct. 2008), 489–511. https://doi.org/10.1007/s10849-008-9067-4Google ScholarDigital Library
- Jennifer Villareale and Jichen Zhu. 2021. Understanding Mental Models of AI through Player-AI Interaction. https://doi.org/10.48550/ARXIV.2103.16168Google ScholarCross Ref
- James C. Walliser, Patrick R. Mead, and Tyler H. Shaw. 2017. The Perception of Teamwork With an Autonomous Agent Enhances Affect and Performance Outcomes. Proceedings of the Human Factors and Ergonomics Society Annual Meeting 61, 1(2017), 231–235. https://doi.org/10.1177/1541931213601541Google ScholarCross Ref
- Qiaosi Wang, Koustuv Saha, Eric Gregori, David Joyner, and Ashok Goel. 2021. Towards Mutual Theory of Mind in Human-AI Interaction: How Language Reflects What Students Perceive About a Virtual Teaching Assistant. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. ACM, Yokohama, Japan, Article 384. https://doi.org/10.1145/3411764.3445645Google ScholarDigital Library
- Michal Čertický and David Churchill. 2021. The Current State of StarCraft AI Competitions and Bots. Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment 13, 2 (June 2021), 2–7. https://ojs.aaai.org/index.php/AIIDE/article/view/12961 Section: Artificial Intelligence for Strategy Games.Google Scholar
Index Terms
- The Hidden Rules of Hanabi: How Humans Outperform AI Agents
Recommendations
Designing Interactions with Intention-Aware Gaze-Enabled Artificial Agents
Human-Computer Interaction – INTERACT 2019AbstractAs it becomes more common for humans to work alongside artificial agents on everyday tasks, it is increasingly important to design artificial agents that can understand and interact with their human counterparts naturally. We posit that an ...
Theory of Mind in Human-AI Interaction
CHI EA '24: Extended Abstracts of the 2024 CHI Conference on Human Factors in Computing SystemsTheory of Mind (ToM), humans’ capability of attributing mental states such as intentions, goals, emotions, and beliefs to ourselves and others, has become a concept of great interest in human-AI interaction research. Given the fundamental role of ToM in ...
Cooperating to Compete: the Mutuality of Cooperation and Competition in Boardgame Play
CHI '18: Proceedings of the 2018 CHI Conference on Human Factors in Computing SystemsThis paper examines the complex relationship between competition and cooperation in boardgame play. We understand boardgaming as distributed cognition, where people work together in a shared activity to accomplish the game. Although players typically ...
Comments