When Should Users Check? Modeling Confirmation Frequency in Multi-Step Agentic AI Tasks
Existing AI agents typically execute multi-step tasks autonomously and only allow user confirmation at the end. During execution, users have little control, making the confirm-at-end approach brittle: a single error can cascade and force a complete restart. Confirming every step avoids such failures, but imposes tedious overhead. Balancing excessive interruptions against costly rollbacks remains an open challenge. We address this problem by modeling confirmation as a minimum time scheduling problem. We conducted a formative study with eight participants, which revealed a recurring Confirmation-Diagnosis-Correction-Redo (CDCR) pattern in how users monitor errors. Based on this pattern, we developed a decision-theoretic model to determine time-efficient confirmation point placement. We then evaluated our approach using a within-subjects study where 48 participants monitored AI agents and repaired their mistakes while executing tasks. Results show that 81 percent of participants preferred our intermediate confirmation approach over the confirm-at-end approach used by existing systems, and task completion time was reduced by 13.54 percent.
Authors
- D. P. Weitekamp (ORCID: https://orcid.org/0000-0003-0079-8000)
- Christopher J. MacLellan (ORCID: https://orcid.org/0000-0003-3084-5189)
Institutions
- Georgia Institute of Technology (US)
Publication Details
- Published
- 2026-04-13
- DOI
- https://doi.org/10.1145/3772318.3790655
- Primary Topic
- Explainable Artificial Intelligence (XAI)
- Type
- article
- Field-Weighted Citation Impact
- 0.00
Funders
- Army Research Laboratory