Automatic Action-Dependency Inference for Selective Revalidation of Tool-Using LLM Agents
Tool-using language-model agents may act on observations that become outdated before a consequential action is committed. Commit-time validators can selectively recheck state, but often assume dependencies or contracts are already specified. This study evaluates whether action-instance dependencies can be inferred from an agent’s own trajectory and how inference errors affect selective revalidation under controlled drift. A deterministic refund benchmark replayed each proposal under no revalidation, FULL_STATE_REVALIDATION (observation-set revalidation plus a current semantic-policy check), LLM-selective revalidation, and oracle-selective revalidation. The action-conditional sample comprised 36 action-bearing captures from 18 represented tasks and 144 matched replay branches; refund-017 and refund-018 remained underfilled with no action-bearing captures. Dependency inference yielded micro precision of 0.8842, recall of 1.0000, and F1 of 0.9385; macro precision was 0.8854, recall 1.0000, and F1 0.9389. Exact-set match was 3/36 (8.3%), with 252 true positives, 33 false positives, zero false negatives, and no observed critical protected-dependency omissions. Because no positive omission cases occurred, the downstream safety effect of actual omissions cannot be estimated. Among 22 invalid-at-commit and 14 valid-at-commit opportunities, LLM-selective revalidation prevented 22/22 unsafe-action opportunities but falsely blocked 5/14 valid actions; task success was 31/36 (86.1%), versus 36/36 for full-state and oracle-selective revalidation. It used 285 reads, 36.9% fewer than full-state (452), and 33 more than oracle-selective (252). These results support feasibility in this controlled setting while leaving precision, predicate resolution, and false blocking unresolved.
Authors
- Habeeb Owoade
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-10-14
- DOI
- https://doi.org/10.5281/zenodo.23363803
- Primary Topic
- AI-based Problem Solving and Planning
- Type
- preprint