Obey, refrain, or contravene: disobedience as a basis for responsible autonomy
Abstract In human-agent teams, there are situations when agents ought to disobey human commands due to various factors, including situational misunderstandings by the command-issuer or rapidly changing conditions. As a reliable member of a team, an agent should be able to evaluate a situation, avoid unnecessary risks, and potentially prevent errors in human judgment stemming from cognitive biases, miscommunication, or incomplete information. Prior work on this topic focuses on disobedience through refraining from execution (i.e., inaction). Here we additionally consider disobedience by contravention—the execution of an alternative course of action, and show that how an agent can disobey affects when it is appropriate for that agent to disobey. To this end, we create a conceptual model of intelligent disobedience. We present results from simulations of our model and discuss the implications of intelligent disobedience in collaborative settings.
Authors
- Gordon Briggs (ORCID: https://orcid.org/0000-0001-8375-3830)
- Munindar Paul Singh (ORCID: https://orcid.org/0000-0003-3599-3893)
- Divya Sundaresan (ORCID: https://orcid.org/0009-0005-2680-0190)
Institutions
- United States Naval Research Laboratory (US)
- North Carolina State University (US)
- United States Navy (US)
Publication Details
- Journal
- Autonomous Agents and Multi-Agent Systems
- Published
- 2026-11-12
- DOI
- https://doi.org/10.1007/s10458-026-09778-0
- Primary Topic
- Human-Automation Interaction and Safety
- Type
- article
- Field-Weighted Citation Impact
- 0.00
Funders
- U.S. Department of Defense
- U.S. Naval Research Laboratory