The Cost of Session Continuation in Multi-Agent Code Generation: A Measurement Study of a Commercial Coding-Agent CLI
A pre-specified measurement study of a commercial coding-agent CLI. We compare fresh sessions (arm B) against write-scope-connected lane continuation (arm C) over seven specifications in one Python repository: 196 runs, 98 paired comparisons. The median paired relative cost difference is +5.51% (BCa 95% interval -3.18% to +10.12%), so the pre-specified criterion of a 20% cost reduction was not met. Runtime and acceptance met their non-inferiority criteria (margins of 15% and 10 percentage points). Post-hoc analysis finds the increase concentrated in one specification, and reports a route that the cost model did not contain: the record of auxiliary-model use separates without exception between fresh and resumed sessions (434 fresh calls versus 84 resumed). This deposit contains the manuscript in Japanese (27 pages) and English (32 pages), the LaTeX sources, and the analysis artifacts. The repository additionally contains the raw measurements of all 196 runs. Reproduction is asymmetric. The analysis re-runs from the bundled data alone. Re-running the experiment additionally requires the commercial service, the execution framework and the repository under test; the latter two carry no license and are deliberately not redistributed. A third party can therefore verify that the same conclusions follow from the bundled data, but cannot independently re-acquire the measurements. This limitation is stated in the manuscript. Licensing is split: MIT for code, CC BY 4.0 for documents and data. See LICENSE, LICENSES/ and REUSE.toml in the repository.
Authors
- Fudo
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-09-08
- DOI
- https://doi.org/10.5281/zenodo.22663556
- Primary Topic
- Software Engineering Research
- Type
- preprint