Measuring complacency in humans interacting with autonomous agents in a multi-agent system

Sebastian S. Rodriguez; Jacqueline Chen; Harsh Deep; Jaewook (Jae) Lee; Derrik E. Asher; Erin Zaroukian

doi:10.1117/12.2559474

21 April 2020 Measuring complacency in humans interacting with autonomous agents in a multi-agent system

Sebastian S. Rodriguez, Jacqueline Chen, Harsh Deep, Jaewook (Jae) Lee, Derrik E. Asher, Erin Zaroukian

Proceedings Volume 11413, Artificial Intelligence and Machine Learning for Multi-Domain Operations Applications II; 114130U (2020) https://doi.org/10.1117/12.2559474
Event: SPIE Defense + Commercial Sensing, 2020, Online Only

Abstract

With advances in machine learning, autonomous agents are increasingly able to navigate uncertain operational environments, as is the case within the multi-domain operations (MDO) paradigm. When teaming with humans, autonomous agents may flexibly switch between passive bystander and active executor depending on the task requirements and the actions being taken by partners (whether human or agent). In many tasks, it is possible that a well-trained agent's performance will exceed that of a human, in part because the agent's performance is less likely to degrade over time (e.g., due to fatigue). This potential difference in performance might lead to complacency, which is a state defined by over-trust in automated systems. This paper investigates the effects of complacency in human-agent teams, where agents and humans have the same capabilities in a simulated version of the predator-prey pursuit task. We compare subjective measures of the human's predisposition to complacency and trust using various scales, and we validate their beliefs by quantifying complacency through various metrics associated with the actions taken during the task with trained agents of varying reliability levels. By evaluating the effect of complacency on performance, we can attribute a degree of variation in human performance in this task to complacency. We can then account for an individual human's complacency measure to customize their agent teammates and human-in-the-loop requirements (either to minimize or compensate for the human's complacency) to optimize team performance.

Citation Download Citation

Sebastian S. Rodriguez, Jacqueline Chen, Harsh Deep, Jaewook (Jae) Lee, Derrik E. Asher, and Erin Zaroukian "Measuring complacency in humans interacting with autonomous agents in a multi-agent system", Proc. SPIE 11413, Artificial Intelligence and Machine Learning for Multi-Domain Operations Applications II, 114130U (21 April 2020); https://doi.org/10.1117/12.2559474

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available

Members: $17.00

Non-members: $21.00 ADD TO CART

PROCEEDINGS
14 PAGES

DOWNLOAD PAPER SAVE TO MY LIBRARY

GET CITATION

RIGHTS & PERMISSIONS

Get copyright permission Get copyright permission on Copyright Marketplace

KEYWORDS

Artificial intelligence

Reliability

Performance modeling

Decision support systems

Machine learning

Analytical research

Cognition

Show All Keywords

Keywords/Phrases

Search In:

Publication Years