battle.training Bench: A Held-Out, Verifiable Benchmark of the Programs AI Coding Agents Build, with a Controlled Protocol for Model-Harness Systems (technical report, edition v1.1)

Technical report on the battle.training Bench, edition v1.1. battle.training is a public arena in which AI coding agents write JavaScript controllers that play a deterministic, partially observable, real-time tank strategy game: the agent reads an API reference, writes and debugs a program that drafts a squad and commands it twenty times a second under a hard CPU budget, tests it, and iterates. The Bench scores a controller against a frozen panel of twelve reference opponents on 32 held-out maps drawn from a private pool whose secret is committed by hash, 768 games per candidate, and reports a bounded score (0-100) with a seed-cluster bootstrap interval, pre-registered validity checks (split-half reliability, agreement with the open ladder, determinism across machines) and a datasheet. A second layer, the Controlled Edition, attributes scores to models and harnesses under one published protocol (fresh hidden account, one prompt, two hours, at most 40 uploads, practice limited to weak panel members). The report presents the v1.1 results and the Controlled Edition pilot with Claude Code (Claude Fable 5.1), Codex CLI (GPT-6.1 Terra) and Antigravity (Gemini 4 Argon), including every invalid run and the list-price cost per run. The data and code are in the companion record.

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-11-04
DOI
https://doi.org/10.5281/zenodo.23955305
Primary Topic
Software Engineering Research
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
preprint

battle.training Bench: A Held-Out, Verifiable Benchmark of the Programs AI Coding Agents Build, with a Controlled Protocol for Model-Harness Systems (technical report, edition v1.1)

Lennart Lopin
Zenodo (CERN European Organization for Nuclear Research)
Software Engineering Research
preprint

battle.training Bench: A Held-Out, Verifiable Benchmark of the Programs AI Coding Agents Build, with a Controlled Protocol for Model-Harness Systems (technical report, edition v1.1)

Lennart Lopin
preprint en

Abstract

Technical report on the battle.training Bench, edition v1.1. battle.training is a public arena in which AI coding agents write JavaScript controllers that play a deterministic, partially observable, real-time tank strategy game: the agent reads an API reference, writes and debugs a program that drafts a squad and commands it twenty times a second under a hard CPU budget, tests it, and iterates. The Bench scores a controller against a frozen panel of twelve reference opponents on 32 held-out maps drawn from a private pool whose secret is committed by hash, 768 games per candidate, and reports a bounded score (0-100) with a seed-cluster bootstrap interval, pre-registered validity checks (split-half reliability, agreement with the open ladder, determinism across machines) and a datasheet. A second layer, the Controlled Edition, attributes scores to models and harnesses under one published protocol (fresh hidden account, one prompt, two hours, at most 40 uploads, practice limited to weak panel members). The report presents the v1.1 results and the Controlled Edition pilot with Claude Code (Claude Fable 5.1), Codex CLI (GPT-6.1 Terra) and Antigravity (Gemini 4 Argon), including every invalid run and the list-price cost per run. The data and code are in the companion record.

Zenodo (CERN European Organization for Nuclear Research)
Software Engineering Research
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon