When does agent shape matter, per instance? A calibrated, abstaining study of coordination-architecture selection

Orchestrators of LLM agents must choose a coordination shape: solve single-agent, fan out and vote, decompose, chain, or run an orchestrator-workers topology. A growing line of work (MasRouter; Google's architecture scaling-laws) treats this as a learnable per-instance decision, assuming task features carry enough signal to route. This paper makes a narrower, more useful claim. In the regime we test, that signal is largely not recoverable from standard features, and the lever that survives is not prediction but calibrated abstention: knowing which decisions to distrust. We are not claiming per-instance routing cannot work; the claim is scoped to this regime (cheap-to-mid models, verifiable tasks) and these standard representations. We test on real (task → per-shape outcome) labels: 159 hard reasoning and coding tasks × 3 model families × 5 shapes, in a regime chosen to favor routing, where shape moves aggregate accuracy and best-of-N voting beats single-agent by twelve points on code. Under the standard task-difficulty features and strong text embeddings we evaluate, a per-instance predictor recovers little: per-feature ROC-AUC runs 0.46 to 0.53, and a trained predictor never clears the always-single majority floor (0.652 vs 0.646), embeddings and a capable LLM reading the full task included. (We do not test the role, cost, and multi-round signals routers also use; the negative is about these standard representations.) Yet a calibrated, abstaining router still extracts value. Escalating the uncertain cases lifts accuracy to 0.76 at a quarter of decisions, a task-bootstrapped interval clear of a random-abstention null, even though the predictor itself never beats the majority class. Ranking which decisions to distrust is easier than naming the winner. The lift is real but modest, and its calibration degrades across model families (ECE 0.0→0.13). As a case study in the same principle, we show one place shape does win decisively, by design rather than selection. A context-chunking architecture holds long-context accuracy where a single agent collapses (single 0.18 vs chunk 0.82 at 16k tokens), reproducing on a second model family. But deploying it is again a calibration problem: the model over-judges when chunking is safe, leaving an oracle-deployable gap (0.92 vs 0.75) that widens on the weaker family as its judge grows more over-optimistic. Across selection, design, and deployment, calibration is the binding constraint.

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-06-29
DOI
https://doi.org/10.5281/zenodo.21030368
Primary Topic
Multi-Agent Systems and Negotiation
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
preprint

When does agent shape matter, per instance? A calibrated, abstaining study of coordination-architecture selection

Benaja Soren OBOUNOU LEKOGO NGUIA
Zenodo (CERN European Organization for Nuclear Research)
Multi-Agent Systems and Negotiation
preprint

When does agent shape matter, per instance? A calibrated, abstaining study of coordination-architecture selection

Benaja Soren OBOUNOU LEKOGO NGUIA
preprint en

Abstract

Orchestrators of LLM agents must choose a coordination shape: solve single-agent, fan out and vote, decompose, chain, or run an orchestrator-workers topology. A growing line of work (MasRouter; Google's architecture scaling-laws) treats this as a learnable per-instance decision, assuming task features carry enough signal to route. This paper makes a narrower, more useful claim. In the regime we test, that signal is largely not recoverable from standard features, and the lever that survives is not prediction but calibrated abstention: knowing which decisions to distrust. We are not claiming per-instance routing cannot work; the claim is scoped to this regime (cheap-to-mid models, verifiable tasks) and these standard representations. We test on real (task → per-shape outcome) labels: 159 hard reasoning and coding tasks × 3 model families × 5 shapes, in a regime chosen to favor routing, where shape moves aggregate accuracy and best-of-N voting beats single-agent by twelve points on code. Under the standard task-difficulty features and strong text embeddings we evaluate, a per-instance predictor recovers little: per-feature ROC-AUC runs 0.46 to 0.53, and a trained predictor never clears the always-single majority floor (0.652 vs 0.646), embeddings and a capable LLM reading the full task included. (We do not test the role, cost, and multi-round signals routers also use; the negative is about these standard representations.) Yet a calibrated, abstaining router still extracts value. Escalating the uncertain cases lifts accuracy to 0.76 at a quarter of decisions, a task-bootstrapped interval clear of a random-abstention null, even though the predictor itself never beats the majority class. Ranking which decisions to distrust is easier than naming the winner. The lift is real but modest, and its calibration degrades across model families (ECE 0.0→0.13). As a case study in the same principle, we show one place shape does win decisively, by design rather than selection. A context-chunking architecture holds long-context accuracy where a single agent collapses (single 0.18 vs chunk 0.82 at 16k tokens), reproducing on a second model family. But deploying it is again a calibration problem: the model over-judges when chunking is safe, leaving an oracle-deployable gap (0.92 vs 0.75) that widens on the weaker family as its judge grows more over-optimistic. Across selection, design, and deployment, calibration is the binding constraint.

Zenodo (CERN European Organization for Nuclear Research)
Multi-Agent Systems and Negotiation
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.