battle.training Bench: A Held-Out, Verifiable Benchmark of the Programs AI Coding Agents Build, with a Controlled Protocol for Model-Harness Systems (technical report, edition v1.1)
Technical report on the battle.training Bench, edition v1.1. battle.training is a public arena in which AI coding agents write JavaScript controllers that play a deterministic, partially observable, real-time tank strategy game: the agent reads an API reference, writes and debugs a program that drafts a squad and commands it twenty times a second under a hard CPU budget, tests it, and iterates. The Bench scores a controller against a frozen panel of twelve reference opponents on 32 held-out maps drawn from a private pool whose secret is committed by hash, 768 games per candidate, and reports a bounded score (0-100) with a seed-cluster bootstrap interval, pre-registered validity checks (split-half reliability, agreement with the open ladder, determinism across machines) and a datasheet. A second layer, the Controlled Edition, attributes scores to models and harnesses under one published protocol (fresh hidden account, one prompt, two hours, at most 40 uploads, practice limited to weak panel members). The report presents the v1.1 results and the Controlled Edition pilot with Claude Code (Claude Fable 5.1), Codex CLI (GPT-6.1 Terra) and Antigravity (Gemini 4 Argon), including every invalid run and the list-price cost per run. The data and code are in the companion record.
Authors
- Lennart Lopin (ORCID: https://orcid.org/0009-0001-2751-2060)
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-11-04
- DOI
- https://doi.org/10.5281/zenodo.23955305
- Primary Topic
- Software Engineering Research
- Type
- preprint