Ledger Vouching: Grounding Someone Else's Agent over the Wire
Tool-using LLM agents ship reports whose load-bearing values — numbers, identifiers, quantities — are frequently fabricated in-head: across two model generations, each measured under its own value-dense reporting demand, the per-token fabrication rate collapsed while the rate of episodes shipping at least one unverified value stayed at 75–81%. We present ledger vouching: an OpenAI-protocol proxy (the ledger voucher), adopted by changing one base_url line, compiles everything it observes on the wire into an observation ledger and deterministically vouches every load-bearing value in its extracted class (digit-bearing tokens: numbers, dates, prices, quantities, alphanumeric identifiers) in an agent's terminal answer to that ledger — no harness cooperation, and no verdict authority for any model: the ledger voucher itself enumerates verified candidate origins by reverse lookup against the ledger, and the model only selects among them for the audit trail. Enforcement is a three-mode ladder — flag, block (honest refusal), and retry (a close-gate keep-alive that pushes the agent to ground its own answer inside the protocol) — with a mandatory explicit fail posture; the guarantee is existence-semantics provenance, never correctness, with its residual refusal families stated and measured. Across two environments and two model generations we measure: transparency from byte level to operational (Δ = 0.000); zero unverified-value shipping in block mode (0/16 and 0/16 in live runs across generations under the enforced predicate, with the refusal cost quantified alongside; 814/814 fabricated-token recall against a privileged oracle in replay); honest, graceful degradation of retry on weak models alongside one-round convergence on a capable one; deterministic cost amplification (1.0×/2.0×/3.0×); and 13% directional agreement with an LLM judge that prefers fabricated answers in 10% of pairs. A companion paper reports that preference training on the same deterministic verifier's labels failed to reduce fabrication while a zero-training environment contract succeeded; the prescription our combined evidence supports is contract plus deterministic enforcement at the boundary.
Authors
- Shin’ya Kouda
Institutions
- DermResearch (United States) (US)
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-09-04
- DOI
- https://doi.org/10.5281/zenodo.22296273
- Primary Topic
- Scientific Computing and Data Management
- Type
- preprint