Exactly Once Across Retries and Restarts: Measuring Duplicate and Stale Side Effects in Tool-Using LLM Agents

When a tool call fails, an agent cannot tell if the action went through. What it reads next, the error or the resume note, often decides whether it repeats a committed step. We hold the committed state of a simulated ledger fixed and change only that text, across 12,733 episodes, six models and four providers. We also count duplicates as a server that honours idempotency keys would see them (post hoc). Replacing a bare "internal server error" with a status saying the outcome is unknown cut duplicates for Haiku 4.5 and Qwen3.6-27B from over 0.6 to at most 0.19 in every wording, with or without keys honoured. For Sonnet 5 and Fable 5 it did no better than a timeout, and honouring keys removes half or more of its effect. Documented API errors that say the outcome is unknown caused fewer duplicates than generic ones. Restarted with only the task, 115/192 sessions redid a committed step with keys honoured (155/192 as run). A two-sentence note that an earlier session was interrupted made every model check the log first, removing nearly all duplicates. On Stripe test mode neither text left duplicates, though rates matched the simulator in under half the cells.

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-11-12
DOI
https://doi.org/10.5281/zenodo.24165777
Primary Topic
Software System Performance and Reliability
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Exactly Once Across Retries and Restarts: Measuring Duplicate and Stale Side Effects in Tool-Using LLM Agents

Sanjana Gopnal Swamy
Zenodo (CERN European Organization for Nuclear Research)
Software System Performance and Reliability
preprint

Exactly Once Across Retries and Restarts: Measuring Duplicate and Stale Side Effects in Tool-Using LLM Agents

Sanjana Gopnal Swamy
preprint en

Abstract

When a tool call fails, an agent cannot tell if the action went through. What it reads next, the error or the resume note, often decides whether it repeats a committed step. We hold the committed state of a simulated ledger fixed and change only that text, across 12,733 episodes, six models and four providers. We also count duplicates as a server that honours idempotency keys would see them (post hoc). Replacing a bare "internal server error" with a status saying the outcome is unknown cut duplicates for Haiku 4.5 and Qwen3.6-27B from over 0.6 to at most 0.19 in every wording, with or without keys honoured. For Sonnet 5 and Fable 5 it did no better than a timeout, and honouring keys removes half or more of its effect. Documented API errors that say the outcome is unknown caused fewer duplicates than generic ones. Restarted with only the task, 115/192 sessions redid a committed step with keys honoured (155/192 as run). A two-sentence note that an earlier session was interrupted made every model check the log first, removing nearly all duplicates. On Stripe test mode neither text left duplicates, though rates matched the simulator in under half the cells.

Zenodo (CERN European Organization for Nuclear Research)
Software System Performance and Reliability
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon