AURA: Evidence-Gated Verification of Task Completion in Multi-Step AI Agents

AI agents complete tasks through workflows involving multiple tool calls and changes to stored data. Verifying completion requires checking whether these actions achieved the requested outcome while respecting the task's constraints. Successful tool execution alone cannot establish this. We present AURA (Agent Uncertainty and Residue Arbiter), a Python and SQLite framework that verifies postconditions, permitted mutations, process requirements, and observation coverage. Independent observations are bound to execution epochs and evaluated under explicit evidence rules to produce PASS, FAIL, or UNCERTAIN verdicts. An optional Counterexample-guided Evaluation with Risk Amortization (CERA) planner selects verification probes using finite candidate worlds, prior weights, and inspection costs; the verifier independently determines the verdict. Evaluation covers 13 scripted local scenarios, 36 controlled SQL cases, and two separate five-episode Qwen2.5 agent cohorts. Across these 59 records, AURA produced 13 PASS, 39 FAIL, and seven UNCERTAIN verdicts. All 59 agreed with supplied harness expectations or specification-sharing reference judgments; this concordance is not independently established verification accuracy. A fresh validation run reproduced these totals through 46 saved-evidence replays and 13 regenerated local scenarios, with replay comparisons limited to selected certificate fields. A separate planning study shows that CERA's modeled inspection-cost advantage depends on prior assumptions and does not hold across all evaluated regimes. The findings support the feasibility of evidence-gated task-completion verification within declared observation boundaries, without establishing general agent reliability, independent specification correctness, or detection of unobserved transient effects.

Authors

Institutions

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-10-30
DOI
https://doi.org/10.5281/zenodo.23800301
Primary Topic
Software Testing and Debugging Techniques
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

AURA: Evidence-Gated Verification of Task Completion in Multi-Step AI Agents

Shanmukh Datta
3 links
Zenodo (CERN European Organization for Nuclear Research)
Software Testing and Debugging Techniques
article

AURA: Evidence-Gated Verification of Task Completion in Multi-Step AI Agents

Shanmukh Datta
article en

Abstract

AI agents complete tasks through workflows involving multiple tool calls and changes to stored data. Verifying completion requires checking whether these actions achieved the requested outcome while respecting the task's constraints. Successful tool execution alone cannot establish this. We present AURA (Agent Uncertainty and Residue Arbiter), a Python and SQLite framework that verifies postconditions, permitted mutations, process requirements, and observation coverage. Independent observations are bound to execution epochs and evaluated under explicit evidence rules to produce PASS, FAIL, or UNCERTAIN verdicts. An optional Counterexample-guided Evaluation with Risk Amortization (CERA) planner selects verification probes using finite candidate worlds, prior weights, and inspection costs; the verifier independently determines the verdict. Evaluation covers 13 scripted local scenarios, 36 controlled SQL cases, and two separate five-episode Qwen2.5 agent cohorts. Across these 59 records, AURA produced 13 PASS, 39 FAIL, and seven UNCERTAIN verdicts. All 59 agreed with supplied harness expectations or specification-sharing reference judgments; this concordance is not independently established verification accuracy. A fresh validation run reproduced these totals through 46 saved-evidence replays and 13 regenerated local scenarios, with replay comparisons limited to selected certificate fields. A separate planning study shows that CERA's modeled inspection-cost advantage depends on prior assumptions and does not hold across all evaluated regimes. The findings support the feasibility of evidence-gated task-completion verification within declared observation boundaries, without establishing general agent reliability, independent specification correctness, or detection of unobserved transient effects.

Zenodo (CERN European Organization for Nuclear Research)
Dr. B. R. Ambedkar National Institute of Technology Jalandhar (IN)
Openalex Percentile: Top 4%
Software Testing and Debugging Techniques
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon
AURA: Evidence-Gated Verification of Task Completion in Multi-Step AI Agents — Shanmukh Datta · Zenodo (CERN European Organization for Nuclear Research) (2026) | TGRS Research Map | TGRS