Structured Runtime Intervention Improves Verification Behaviour in LLM Agents: A CRepair Pilot Study

We present a pilot study testing whether runtime coherence enforcement improves structural self-repair in LLM agents. Building on the CRepair benchmark (Kaminovs 2025d), we introduce the CRepair Wrapper — a thin enforcement layer that intercepts agent responses, scores them on the Cₛₑₚₐᵢᵣ = D × R × V × S metric, and applies targeted repair instructions when the loop is not closed. Across 3 independent runs (39 scenario pairs each, randomly ordered in runs 2–3), the wrapper consistently improved mean Cₛₑₚₐᵢᵣ from 0.474 to 0.808 (ΔC = +0.333), compared to only +0.051 for a generic retry control. Verification traces (V > 0) were observed in all wrapped responses, compared with 54–77% at baseline. Critically, the wrapped condition was the only condition to eliminate zero-component failures across all three runs. We report the 0.808 ceiling as a structural artefact of the current judge’s V scoring distribution, not a claim of true reasoning verification. The central finding is that structured repair loops, not mere re-prompting, drive the improvement — providing preliminary evidence that runtime coherence enforcement is a viable and distinct approach to improving LLM agent reliability.

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-05-21
DOI
https://doi.org/10.5281/zenodo.20324351
Primary Topic
Modular Robots and Swarm Intelligence
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
preprint

Structured Runtime Intervention Improves Verification Behaviour in LLM Agents: A CRepair Pilot Study

Sergejs Kaminovs
Zenodo (CERN European Organization for Nuclear Research)
Modular Robots and Swarm Intelligence
preprint

Structured Runtime Intervention Improves Verification Behaviour in LLM Agents: A CRepair Pilot Study

Sergejs Kaminovs
preprint en

Abstract

We present a pilot study testing whether runtime coherence enforcement improves structural self-repair in LLM agents. Building on the CRepair benchmark (Kaminovs 2025d), we introduce the CRepair Wrapper — a thin enforcement layer that intercepts agent responses, scores them on the Cₛₑₚₐᵢᵣ = D × R × V × S metric, and applies targeted repair instructions when the loop is not closed. Across 3 independent runs (39 scenario pairs each, randomly ordered in runs 2–3), the wrapper consistently improved mean Cₛₑₚₐᵢᵣ from 0.474 to 0.808 (ΔC = +0.333), compared to only +0.051 for a generic retry control. Verification traces (V > 0) were observed in all wrapped responses, compared with 54–77% at baseline. Critically, the wrapped condition was the only condition to eliminate zero-component failures across all three runs. We report the 0.808 ceiling as a structural artefact of the current judge’s V scoring distribution, not a claim of true reasoning verification. The central finding is that structured repair loops, not mere re-prompting, drive the improvement — providing preliminary evidence that runtime coherence enforcement is a viable and distinct approach to improving LLM agent reliability.

Zenodo (CERN European Organization for Nuclear Research)
Modular Robots and Swarm Intelligence
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.