ASSAY: No Action Without a Prediction

ASSAY is a general agent harness built so that an LLM agent reasons its way through an unfamiliar world, learns that world from interaction at test time, and carries what it learns into later runs. A world is attached through one small adapter and a registry of permitted actions, and the agent is never told what its actions do: it discovers each by predicting its effect and paying for the answer. The single governing rule is that there is no action without a prediction, and every prediction is graded in code against the world's own response. That discipline is what forces the agent to build a world model under scarcity, which is where the reasoning and the test-time learning come from. The agent internalizes what it learns through self-declared instruments over its observations and through knowledge that carries across runs, demoted to hypothesis until re-earned, a tested capability whose measurement is the next phase. A by-product of the same discipline, not its aim, is a fully auditable record: every event lands on a hash-chained journal that a standalone public tool re-verifies from the artifacts alone, and a single ungated action invalidates a run. We evaluate the same harness on three worlds. On the ARC-AGI-3 public set it scores RHAE 96.54, confirmed by the benchmark server on a public scorecard, winning 24 of 25 games under hard action caps at a measured 8.0% exploration overhead and approximately zero capability cost over completed games. On the Factorio Learning Environment the same design fail-closes the sanctioned-interface bypass a published agent fell to. On the OOLONG long-context benchmark an early sweep places it in the same band as mature general harnesses across a 128K to 4M token ladder. The ARC-AGI-3 campaign also produced an epistemic finding: on six occasions the agent proved a level impossible, each proof consistent with every recorded transition and wrong exactly where none had gone, and a three-step audit of each proof's unexercised rules converted five into wins within existing budgets.

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-08-28
DOI
https://doi.org/10.5281/zenodo.22145967
Primary Topic
Artificial Intelligence in Games
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

ASSAY: No Action Without a Prediction

Arjmandi Mohsen
Zenodo (CERN European Organization for Nuclear Research)
Artificial Intelligence in Games
article

ASSAY: No Action Without a Prediction

Arjmandi Mohsen
article en

Abstract

ASSAY is a general agent harness built so that an LLM agent reasons its way through an unfamiliar world, learns that world from interaction at test time, and carries what it learns into later runs. A world is attached through one small adapter and a registry of permitted actions, and the agent is never told what its actions do: it discovers each by predicting its effect and paying for the answer. The single governing rule is that there is no action without a prediction, and every prediction is graded in code against the world's own response. That discipline is what forces the agent to build a world model under scarcity, which is where the reasoning and the test-time learning come from. The agent internalizes what it learns through self-declared instruments over its observations and through knowledge that carries across runs, demoted to hypothesis until re-earned, a tested capability whose measurement is the next phase. A by-product of the same discipline, not its aim, is a fully auditable record: every event lands on a hash-chained journal that a standalone public tool re-verifies from the artifacts alone, and a single ungated action invalidates a run. We evaluate the same harness on three worlds. On the ARC-AGI-3 public set it scores RHAE 96.54, confirmed by the benchmark server on a public scorecard, winning 24 of 25 games under hard action caps at a measured 8.0% exploration overhead and approximately zero capability cost over completed games. On the Factorio Learning Environment the same design fail-closes the sanctioned-interface bypass a published agent fell to. On the OOLONG long-context benchmark an early sweep places it in the same band as mature general harnesses across a 128K to 4M token ladder. The ARC-AGI-3 campaign also produced an epistemic finding: on six occasions the agent proved a level impossible, each proof consistent with every recorded transition and wrong exactly where none had gone, and a three-step audit of each proof's unexercised rules converted five into wins within existing budgets.

Zenodo (CERN European Organization for Nuclear Research)
Openalex Percentile: Top 8%
Artificial Intelligence in Games
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.