Proof-Carrying Operation: Device Agents Whose Completion Claims Are Checked Against the Screen

Agents that operate phones decide for themselves when a task is done, and that decision is where they fail: they stop with sub-goals unfinished, give up on facts that are hard to find, and report values they never saw. Proof-carrying operation (PCO) takes the decision away from the model. The agent compiles the goal into typed obligations (state changes to make, facts to report, results to derive, conditions to resolve) and must close each with the value it claims and the recorded screen that shows it. A deterministic checker with no app knowledge accepts a claim only if the cited screen's accessibility tree contains it, challenges every claim of absence, refuses to end the episode while obligations are open, and completes the final answer only from claims it accepted. Every run leaves a certificate. On iOSWorld, PCO raises performance from 77 to 90 of 133 tasks fully passed (57.9% to 67.7%) and the mean rubric score from 0.857 to 0.885 (exact McNemar p = 0.007); a replicate run lands at 91/133 (0.890), without app-specific knowledge. The gain sits in multi-app tasks (25 to 36 of 60) and in criteria about actions on the device, not in answer wording. An ablation without resumption rounds keeps most of the gain (86/133), which locates the effect in the obligation ledger the agent must keep rather than in being sent back. We audit the checker itself: of 60 randomly sampled accepted claims, 48 sound, 5 editor-state, 4 weak and 3 mismatched; every ledger, screen and certificate is published so that the audit can be repeated. On AndroidWorld, near its ceiling, the same operator averages 95.1% pass@1 over three suite seeds with no app knowledge. Code: https://github.com/verdikt-ai/verdikt. Every run, frame and statistic: https://verdikt-ai.github.io/verdikt/

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-10-13
DOI
https://doi.org/10.5281/zenodo.23349257
Primary Topic
Software Testing and Debugging Techniques
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
preprint

Proof-Carrying Operation: Device Agents Whose Completion Claims Are Checked Against the Screen

Kedar Haldankar, Hussein Abdinoor Mohamed
Zenodo (CERN European Organization for Nuclear Research)
Software Testing and Debugging Techniques
preprint

Proof-Carrying Operation: Device Agents Whose Completion Claims Are Checked Against the Screen

Kedar Haldankar, Hussein Abdinoor Mohamed
preprint en

Abstract

Agents that operate phones decide for themselves when a task is done, and that decision is where they fail: they stop with sub-goals unfinished, give up on facts that are hard to find, and report values they never saw. Proof-carrying operation (PCO) takes the decision away from the model. The agent compiles the goal into typed obligations (state changes to make, facts to report, results to derive, conditions to resolve) and must close each with the value it claims and the recorded screen that shows it. A deterministic checker with no app knowledge accepts a claim only if the cited screen's accessibility tree contains it, challenges every claim of absence, refuses to end the episode while obligations are open, and completes the final answer only from claims it accepted. Every run leaves a certificate. On iOSWorld, PCO raises performance from 77 to 90 of 133 tasks fully passed (57.9% to 67.7%) and the mean rubric score from 0.857 to 0.885 (exact McNemar p = 0.007); a replicate run lands at 91/133 (0.890), without app-specific knowledge. The gain sits in multi-app tasks (25 to 36 of 60) and in criteria about actions on the device, not in answer wording. An ablation without resumption rounds keeps most of the gain (86/133), which locates the effect in the obligation ledger the agent must keep rather than in being sent back. We audit the checker itself: of 60 randomly sampled accepted claims, 48 sound, 5 editor-state, 4 weak and 3 mismatched; every ledger, screen and certificate is published so that the audit can be repeated. On AndroidWorld, near its ceiling, the same operator averages 95.1% pass@1 over three suite seeds with no app knowledge. Code: https://github.com/verdikt-ai/verdikt. Every run, frame and statistic: https://verdikt-ai.github.io/verdikt/

Zenodo (CERN European Organization for Nuclear Research)
Software Testing and Debugging Techniques
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.