Obey, refrain, or contravene: disobedience as a basis for responsible autonomy

Abstract In human-agent teams, there are situations when agents ought to disobey human commands due to various factors, including situational misunderstandings by the command-issuer or rapidly changing conditions. As a reliable member of a team, an agent should be able to evaluate a situation, avoid unnecessary risks, and potentially prevent errors in human judgment stemming from cognitive biases, miscommunication, or incomplete information. Prior work on this topic focuses on disobedience through refraining from execution (i.e., inaction). Here we additionally consider disobedience by contravention—the execution of an alternative course of action, and show that how an agent can disobey affects when it is appropriate for that agent to disobey. To this end, we create a conceptual model of intelligent disobedience. We present results from simulations of our model and discuss the implications of intelligent disobedience in collaborative settings.

Authors

Institutions

Publication Details

Journal
Autonomous Agents and Multi-Agent Systems
Published
2026-11-12
DOI
https://doi.org/10.1007/s10458-026-09778-0
Primary Topic
Human-Automation Interaction and Safety
Type
article
Field-Weighted Citation Impact
0.00

Funders

Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Obey, refrain, or contravene: disobedience as a basis for responsible autonomy

Gordon Briggs, Munindar Paul Singh, Divya Sundaresan
Autonomous Agents and Multi-Agent Systems
Human-Automation Interaction and Safety
article

Obey, refrain, or contravene: disobedience as a basis for responsible autonomy

Gordon Briggs, Munindar Paul Singh, Divya Sundaresan
article en

Abstract

Abstract In human-agent teams, there are situations when agents ought to disobey human commands due to various factors, including situational misunderstandings by the command-issuer or rapidly changing conditions. As a reliable member of a team, an agent should be able to evaluate a situation, avoid unnecessary risks, and potentially prevent errors in human judgment stemming from cognitive biases, miscommunication, or incomplete information. Prior work on this topic focuses on disobedience through refraining from execution (i.e., inaction). Here we additionally consider disobedience by contravention—the execution of an alternative course of action, and show that how an agent can disobey affects when it is appropriate for that agent to disobey. To this end, we create a conceptual model of intelligent disobedience. We present results from simulations of our model and discuss the implications of intelligent disobedience in collaborative settings.

Autonomous Agents and Multi-Agent SystemsVol. 40(2)
United States Naval Research Laboratory (US), North Carolina State University (US), United States Navy (US)
U.S. Department of Defense, U.S. Naval Research Laboratory
Human-Automation Interaction and Safety
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon