Automatic Action-Dependency Inference for Selective Revalidation of Tool-Using LLM Agents

Tool-using language-model agents may act on observations that become outdated before a consequential action is committed. Commit-time validators can selectively recheck state, but often assume dependencies or contracts are already specified. This study evaluates whether action-instance dependencies can be inferred from an agent’s own trajectory and how inference errors affect selective revalidation under controlled drift. A deterministic refund benchmark replayed each proposal under no revalidation, FULL_STATE_REVALIDATION (observation-set revalidation plus a current semantic-policy check), LLM-selective revalidation, and oracle-selective revalidation. The action-conditional sample comprised 36 action-bearing captures from 18 represented tasks and 144 matched replay branches; refund-017 and refund-018 remained underfilled with no action-bearing captures. Dependency inference yielded micro precision of 0.8842, recall of 1.0000, and F1 of 0.9385; macro precision was 0.8854, recall 1.0000, and F1 0.9389. Exact-set match was 3/36 (8.3%), with 252 true positives, 33 false positives, zero false negatives, and no observed critical protected-dependency omissions. Because no positive omission cases occurred, the downstream safety effect of actual omissions cannot be estimated. Among 22 invalid-at-commit and 14 valid-at-commit opportunities, LLM-selective revalidation prevented 22/22 unsafe-action opportunities but falsely blocked 5/14 valid actions; task success was 31/36 (86.1%), versus 36/36 for full-state and oracle-selective revalidation. It used 285 reads, 36.9% fewer than full-state (452), and 33 more than oracle-selective (252). These results support feasibility in this controlled setting while leaving precision, predicate resolution, and false blocking unresolved.

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-10-14
DOI
https://doi.org/10.5281/zenodo.23363803
Primary Topic
AI-based Problem Solving and Planning
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
preprint

Automatic Action-Dependency Inference for Selective Revalidation of Tool-Using LLM Agents

Habeeb Owoade
Zenodo (CERN European Organization for Nuclear Research)
AI-based Problem Solving and Planning
preprint

Automatic Action-Dependency Inference for Selective Revalidation of Tool-Using LLM Agents

Habeeb Owoade
preprint en

Abstract

Tool-using language-model agents may act on observations that become outdated before a consequential action is committed. Commit-time validators can selectively recheck state, but often assume dependencies or contracts are already specified. This study evaluates whether action-instance dependencies can be inferred from an agent’s own trajectory and how inference errors affect selective revalidation under controlled drift. A deterministic refund benchmark replayed each proposal under no revalidation, FULL_STATE_REVALIDATION (observation-set revalidation plus a current semantic-policy check), LLM-selective revalidation, and oracle-selective revalidation. The action-conditional sample comprised 36 action-bearing captures from 18 represented tasks and 144 matched replay branches; refund-017 and refund-018 remained underfilled with no action-bearing captures. Dependency inference yielded micro precision of 0.8842, recall of 1.0000, and F1 of 0.9385; macro precision was 0.8854, recall 1.0000, and F1 0.9389. Exact-set match was 3/36 (8.3%), with 252 true positives, 33 false positives, zero false negatives, and no observed critical protected-dependency omissions. Because no positive omission cases occurred, the downstream safety effect of actual omissions cannot be estimated. Among 22 invalid-at-commit and 14 valid-at-commit opportunities, LLM-selective revalidation prevented 22/22 unsafe-action opportunities but falsely blocked 5/14 valid actions; task success was 31/36 (86.1%), versus 36/36 for full-state and oracle-selective revalidation. It used 285 reads, 36.9% fewer than full-state (452), and 33 more than oracle-selective (252). These results support feasibility in this controlled setting while leaving precision, predicate resolution, and false blocking unresolved.

Zenodo (CERN European Organization for Nuclear Research)
AI-based Problem Solving and Planning
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.