Structured Runtime Intervention Improves Verification Behaviour in LLM Agents: A CRepair Pilot Study
We present a pilot study testing whether runtime coherence enforcement improves structural self-repair in LLM agents. Building on the CRepair benchmark (Kaminovs 2025d), we introduce the CRepair Wrapper — a thin enforcement layer that intercepts agent responses, scores them on the Cₛₑₚₐᵢᵣ = D × R × V × S metric, and applies targeted repair instructions when the loop is not closed. Across 3 independent runs (39 scenario pairs each, randomly ordered in runs 2–3), the wrapper consistently improved mean Cₛₑₚₐᵢᵣ from 0.474 to 0.808 (ΔC = +0.333), compared to only +0.051 for a generic retry control. Verification traces (V > 0) were observed in all wrapped responses, compared with 54–77% at baseline. Critically, the wrapped condition was the only condition to eliminate zero-component failures across all three runs. We report the 0.808 ceiling as a structural artefact of the current judge’s V scoring distribution, not a claim of true reasoning verification. The central finding is that structured repair loops, not mere re-prompting, drive the improvement — providing preliminary evidence that runtime coherence enforcement is a viable and distinct approach to improving LLM agent reliability.
Authors
- Sergejs Kaminovs
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-05-21
- DOI
- https://doi.org/10.5281/zenodo.20324352
- Primary Topic
- Modular Robots and Swarm Intelligence
- Type
- preprint