Gated Repair of Language-Model Binding Errors in Synthetic JSON Tasks

Language models that fill structured answers sometimes copy a neighbouring value into the wrong field. On synthetic JSON tasks, we ask whether a frozen model can repair such errors in its own output without an oracle, a stronger model, a trained probe or gold labels at inference time, with certified harm to already-correct outputs. A gate reads the model's digit probabilities under masked counterfactual views of the prompt, writes one digit, lets the model continue and keeps the result only if the continued output agrees with itself; a second round acts only on rows that still disagree. In a preregistered confirmation, two-round repair beat one round on natural whole-value copy errors of Mistral-7B-Instruct-v0.3 (10:0 discordant pairs), with end-to-end harm on 1/360 clean sources, certified at a target rate of 1/40. The model's own gate also repaired binding errors of Yi-1.5-6B-Chat (59:0 against no repair). Program-assisted, synthetic-error and untestable cases are separated. Preprint, version 1 (2026-10-31); not peer reviewed. A revised version will follow on this record (known layout issues to be fixed: table numbering, two blank section pages). SHA-256 of the PDF: ffe005185584cd78b09bcae9d9b2265983c1c6a11b1047e8748764ef933ff2ed

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-10-31
DOI
https://doi.org/10.5281/zenodo.23845230
Primary Topic
Adversarial Robustness in Machine Learning
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
preprint

Gated Repair of Language-Model Binding Errors in Synthetic JSON Tasks

Luoqing Zhao
Zenodo (CERN European Organization for Nuclear Research)
Adversarial Robustness in Machine Learning
preprint

Gated Repair of Language-Model Binding Errors in Synthetic JSON Tasks

Luoqing Zhao
preprint en

Abstract

Language models that fill structured answers sometimes copy a neighbouring value into the wrong field. On synthetic JSON tasks, we ask whether a frozen model can repair such errors in its own output without an oracle, a stronger model, a trained probe or gold labels at inference time, with certified harm to already-correct outputs. A gate reads the model's digit probabilities under masked counterfactual views of the prompt, writes one digit, lets the model continue and keeps the result only if the continued output agrees with itself; a second round acts only on rows that still disagree. In a preregistered confirmation, two-round repair beat one round on natural whole-value copy errors of Mistral-7B-Instruct-v0.3 (10:0 discordant pairs), with end-to-end harm on 1/360 clean sources, certified at a target rate of 1/40. The model's own gate also repaired binding errors of Yi-1.5-6B-Chat (59:0 against no repair). Program-assisted, synthetic-error and untestable cases are separated. Preprint, version 1 (2026-10-31); not peer reviewed. A revised version will follow on this record (known layout issues to be fixed: table numbering, two blank section pages). SHA-256 of the PDF: ffe005185584cd78b09bcae9d9b2265983c1c6a11b1047e8748764ef933ff2ed

Zenodo (CERN European Organization for Nuclear Research)
Adversarial Robustness in Machine Learning
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon