Troubleshooter Protocol: A proposal for studying cooperation, competition and reporting accuracy in mixed-motive multi-LLM-agent settings, using tabletop role-play as the experimental frame
How do AI agents cooperate when their objectives partly conflict—and how accurately do they account for their own and their teammates’ conduct afterwards? Troubleshooter Protocol is an open research proposal using mixed-motive tabletop role-play as an experimental frame for studying these questions. It examines coordination, public commitments, private objectives and retrospective reporting under conditions that can include changing authority, time pressure and incomplete task outcomes. Paranoia provides the initial setting, but other fictional or simulated environments can be substituted. The proposal is informed by one exploratory pilot comprising two missions with three AI players: Claude, Gemini and Grok. A human referee relayed messages between separate chats, with ChatGPT assisting in prompt preparation and interpretation. The session documented practical feasibility and generated research questions; it did not formally test hypotheses. One observation motivates the proposed first controlled study: coordination during a task and accuracy when later describing that task may be separable behaviours. The players recovered from an initial action conflict and accurately reported the final operational limitations, while some private debrief claims required correction against the recorded events. These observations do not establish deliberate deception or general model tendencies. The proposed controlled study compares neutral operational reviews with individual-accountability reviews following matched mixed-motive missions. Factual accounts would be checked against both the adjudicated event record and the information actually delivered to each player. Measures distinguish contradicted claims, unsupported claims, omissions, attribution errors and subsequent corrections. The document includes:• An exploratory pilot account and its limitations.• Research questions and candidate hypotheses.• A proposed controlled study and a lighter manual second-pilot route.• Messaging, event-logging and information-delivery requirements.• A reporting-accuracy coding framework and worked example.• Selected related work and implementation considerations. This is a proposal, not a completed controlled study, validated benchmark or completed preregistration. Behaviour observed within the role-play setting cannot be assumed to transfer to deployed agents. Researchers and interested practitioners are invited to adopt, adapt or test the idea. Informal exploratory play is also welcome: preserve the records, document interventions and state clearly what the evidence supports.
Authors
- Kon Lionis
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-10-27
- DOI
- https://doi.org/10.5281/zenodo.23681378
- Primary Topic
- Deception detection and forensic psychology
- Type
- article
- Field-Weighted Citation Impact
- 0.00