When AI Agents Talk, What Information Gets Lost?

When many AI systems work on the same problem, it is intuitive to think that letting them discuss their answers will naturally make the final result better. But this discussion can also cause useful details to disappear as the systems converge on one shared response. I tested this possibility in long-form policy analysis, where it is necessary for a good answer to keep track of many facts and trade-offs. I built CA-TradeoffEval, an evaluation testbed based on official California ballot-measure materials, and broke ten policy measures into 120 small reference claims and 367 smaller information components. I compared four ways of producing a final answer: one AI agent working alone, three independent agents whose work was combined, ordinary multi-agent debate, and debate in which one participant acted as a critic whose job was to catch missing or weakened information by referring back to the source. Across 160 final outputs from GPT-5.6 Sol, critic-guided debate preserved more source-grounded information than ordinary debate by 3.13 percentage points (95% measure-level bootstrap CI [0.53, 6.34]). Independent aggregation had a smaller positive estimate over a single agent, and ordinary debate was almost indistinguishable from independent aggregation. I then ran a smaller replication using Claude Sonnet 5 on five randomly selected measures. Claude had a similar critic effect (+2.62 points), but the confidence interval was wider and included zero. The Claude replication also showed more distortion in critic debate, suggesting that there is a possible trade-off between keeping information but also possibly introducing more inaccuracies. Overall, the results suggest that how AI agents are organized matters, making them discuss is not enough, and a clearly defined critic role can help preserve information, but it should also be paired with checks for distortion.

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-09-11
DOI
https://doi.org/10.5281/zenodo.22715133
Primary Topic
Ethics and Social Impacts of AI
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
preprint

When AI Agents Talk, What Information Gets Lost?

Spriha Pradeep
Zenodo (CERN European Organization for Nuclear Research)
Ethics and Social Impacts of AI
preprint

When AI Agents Talk, What Information Gets Lost?

Spriha Pradeep
preprint en

Abstract

When many AI systems work on the same problem, it is intuitive to think that letting them discuss their answers will naturally make the final result better. But this discussion can also cause useful details to disappear as the systems converge on one shared response. I tested this possibility in long-form policy analysis, where it is necessary for a good answer to keep track of many facts and trade-offs. I built CA-TradeoffEval, an evaluation testbed based on official California ballot-measure materials, and broke ten policy measures into 120 small reference claims and 367 smaller information components. I compared four ways of producing a final answer: one AI agent working alone, three independent agents whose work was combined, ordinary multi-agent debate, and debate in which one participant acted as a critic whose job was to catch missing or weakened information by referring back to the source. Across 160 final outputs from GPT-5.6 Sol, critic-guided debate preserved more source-grounded information than ordinary debate by 3.13 percentage points (95% measure-level bootstrap CI [0.53, 6.34]). Independent aggregation had a smaller positive estimate over a single agent, and ordinary debate was almost indistinguishable from independent aggregation. I then ran a smaller replication using Claude Sonnet 5 on five randomly selected measures. Claude had a similar critic effect (+2.62 points), but the confidence interval was wider and included zero. The Claude replication also showed more distortion in critic debate, suggesting that there is a possible trade-off between keeping information but also possibly introducing more inaccuracies. Overall, the results suggest that how AI agents are organized matters, making them discuss is not enough, and a clearly defined critic role can help preserve information, but it should also be paired with checks for distortion.

Zenodo (CERN European Organization for Nuclear Research)
Peace, Justice and strong institutions
Ethics and Social Impacts of AI
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.