Safety in numbers: Tackling the deliberative, performative, and infrastructural roots of multi‐agent misalignment
Abstract Multi‐agent systems (MAS)—a uniquely important subset of the wider field commonly called “Agentic AI”—enable complex work processes involving distributed networks of multiple AIs, people, and technology tools. However, they also carry big risks. Root causes of catastrophic safety failures in multi‐agent collaboration (as in human collaboration) can be classed into three overlapping categories: 1. Deliberative —deviating from shared MAS commitments to fulfill explicitly expressed joint intentions; 2. Performative —inability to coordinate well or to adapt to changing circumstances; and 3. Infrastructural —safety, reliability, and trust in the operating environment. In the spirit of the international Multi‐Agent System Safety and Teamwork (MASST) initiative (Bradshaw et al., 2026), our multi‐disciplinary research team seeks to tackle the root causes of these three challenges through development, experimentation, and deployment of an integrated multi‐layered safety pipeline, enabling “safety in numbers” as increasingly powerful multi‐agent systems are deployed. Although this proposal is merely an initial sketch, it is our hope that sharing these ideas will inspire others to contribute to the discussion as we continue moving forward. We look forward not only to exciting discoveries within laboratory settings while also being committed to improving safe rollout of large‐scale MAS into the world to perform work in high‐consequence applications.
Authors
- Larry Bunch
- Rebecca Montanari (ORCID: https://orcid.org/0000-0002-3687-0361)
- Kristen Brent Venable (ORCID: https://orcid.org/0000-0002-1092-9759)
- William de Beaumont
- Jeffrey M. Bradshaw (ORCID: https://orcid.org/0000-0003-0110-8104)
- Jurriaan van Diggelen (ORCID: https://orcid.org/0000-0001-8216-1286)
- Michael T. Goodrich (ORCID: https://orcid.org/0000-0002-8943-191X)
- David D. Woods (ORCID: https://orcid.org/0000-0003-3131-2894)
- Niranjan Suri (ORCID: https://orcid.org/0000-0003-2982-8218)
- Nicolò Romandini (ORCID: https://orcid.org/0000-0002-2820-5978)
- Roberto Fronteddu (ORCID: https://orcid.org/0009-0002-8289-2948)
- James L. Lott
- Jane Pinelis
- Patrick J. Hayes (ORCID: https://orcid.org/0000-0002-6639-9187)
- Brian A. Jalaian (ORCID: https://orcid.org/0000-0003-3029-601X)
- Andrzej Uszok (ORCID: https://orcid.org/0009-0008-4183-5596)
- Matthew Johnson (ORCID: https://orcid.org/0000-0002-3525-695X)
- Sara Cavicchi (ORCID: https://orcid.org/0009-0002-0771-0067)
- Manuele Tele Junior Fernandez (ORCID: https://orcid.org/0009-0006-6954-6505)
- Alessandro Amato (ORCID: https://orcid.org/0009-0005-1209-7166)
- Margaret Warren (ORCID: https://orcid.org/0000-0002-6680-2431)
- Ben Shinnick
- Mica Vignati
Institutions
- Brigham Young University (US)
- Florida Institute for Human and Machine Cognition (US)
- Johns Hopkins University (US)
- Johns Hopkins University Applied Physics Laboratory (US)
- The Ohio State University (US)
- University of Bologna (IT)
Publication Details
- Journal
- AI Magazine
- Published
- 2026-11-08
- DOI
- https://doi.org/10.1002/aaai.70124
- Primary Topic
- Multi-Agent Systems and Negotiation
- Type
- article
- Field-Weighted Citation Impact
- 0.00
Funders
- Stanford Bio-X