Proof of Intended Outcome: Intent-Anchored Verification of Autonomous Agent Completion Claims

When an autonomous agent reports that a task is done, the systems around it usually check the report against the agent's own description of what it did. If the executor deployed to the wrong hostname, refunded the wrong order or migrated the wrong database, every one of those checks passes. This paper proposes Proof of Intended Outcome (PIO), a framework that treats every completion report as a claim to be tested against the user's intended outcome, using requirements fixed before execution, evidence bound to exact targets, decisive handling of mandatory violations, dependency-aware aggregation of evidence, and an explicit validity window. It separates five ways a completion can be wrong while looking right (intent fidelity, specification correctness, outcome correctness, evidence reliability, temporal validity), gives a domain-independent data model for intents, requirements, claims, evidence and attestations, and connects the framework to the confirmation ladder and doubt signal of the author's earlier work. Findings. Fault-injection study of LLM verifiers (8 tasks in 8 domains, 5 injected fault classes, 5 information arms; Claude Haiku 5.5 ×3 and Claude Sonnet 5.5 ×1; 960 verdicts for 1.14 USD): verifiers given only the executor's report never caught a wrong-target completion whose clean counterpart they passed (0/11 and 0/6 pairs) and accepted 18% and 22% of all faulty completions. Giving the same verifiers the user's original request cut false acceptance to 2% and caught 21/24 and 7/8 wrong targets; the only misses were a genuinely ambiguous restaurant name. Explicit evaluation rules added little on top of the request. Letting the verifier derive its own requirements kept false acceptance at 0–2% but rejected 75% and 88% of correct completions: derived specifications grew from 3 to 5.6–7.0 requirements, mostly process conditions the user never asked for. Simulation: seven verifiers that agree because they read one source report P(success) = 0.99999994 when the true rate is 0.939, and at a 0.99 bar admit 14.5% of failed tasks; a dependency-aware posterior admits none. Practical upshot: route the original request, not the executor's summary, to the verifier, and fix requirements before the executor runs. Contents: manuscript (PDF) and artifact archive with scenario generator, verifier harness, every cached model response, analysis scripts and the correlated-evidence simulation. Code MIT; text CC BY 4.0.

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-10-28
DOI
https://doi.org/10.5281/zenodo.23733141
Primary Topic
Software Testing and Debugging Techniques
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
preprint

Proof of Intended Outcome: Intent-Anchored Verification of Autonomous Agent Completion Claims

Sterling Morrison
5 links
Zenodo (CERN European Organization for Nuclear Research)
Software Testing and Debugging Techniques
preprint

Proof of Intended Outcome: Intent-Anchored Verification of Autonomous Agent Completion Claims

Sterling Morrison
preprint en

Abstract

When an autonomous agent reports that a task is done, the systems around it usually check the report against the agent's own description of what it did. If the executor deployed to the wrong hostname, refunded the wrong order or migrated the wrong database, every one of those checks passes. This paper proposes Proof of Intended Outcome (PIO), a framework that treats every completion report as a claim to be tested against the user's intended outcome, using requirements fixed before execution, evidence bound to exact targets, decisive handling of mandatory violations, dependency-aware aggregation of evidence, and an explicit validity window. It separates five ways a completion can be wrong while looking right (intent fidelity, specification correctness, outcome correctness, evidence reliability, temporal validity), gives a domain-independent data model for intents, requirements, claims, evidence and attestations, and connects the framework to the confirmation ladder and doubt signal of the author's earlier work. Findings. Fault-injection study of LLM verifiers (8 tasks in 8 domains, 5 injected fault classes, 5 information arms; Claude Haiku 5.5 ×3 and Claude Sonnet 5.5 ×1; 960 verdicts for 1.14 USD): verifiers given only the executor's report never caught a wrong-target completion whose clean counterpart they passed (0/11 and 0/6 pairs) and accepted 18% and 22% of all faulty completions. Giving the same verifiers the user's original request cut false acceptance to 2% and caught 21/24 and 7/8 wrong targets; the only misses were a genuinely ambiguous restaurant name. Explicit evaluation rules added little on top of the request. Letting the verifier derive its own requirements kept false acceptance at 0–2% but rejected 75% and 88% of correct completions: derived specifications grew from 3 to 5.6–7.0 requirements, mostly process conditions the user never asked for. Simulation: seven verifiers that agree because they read one source report P(success) = 0.99999994 when the true rate is 0.939, and at a 0.99 bar admit 14.5% of failed tasks; a dependency-aware posterior admits none. Practical upshot: route the original request, not the executor's summary, to the verifier, and fix requirements before the executor runs. Contents: manuscript (PDF) and artifact archive with scenario generator, verifier harness, every cached model response, analysis scripts and the correlated-evidence simulation. Code MIT; text CC BY 4.0.

Zenodo (CERN European Organization for Nuclear Research)
Software Testing and Debugging Techniques
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon