When Correct Answers Carry Invalid State: Evaluating Explicit State Carriers in Stateful LLM Agents

Stateful LLM agents often expose bounded state for later computation. A correct final answer does not establish that this state is safe, sufficient for continuation, or aligned with the answer. We operationalize this acceptance question with a task-relative, field-sensitive, representation-normalized contract for explicit state carriers. Deterministic adapters map native summary, visible-carry, and typed-state representations into a common claimed-state layer; the contract then evaluates parseability, actionable-state presence, exclusion safety, operational sufficiency, and answer–state consistency. We apply the contract to 7,200 frozen outputs over 40 tasks, five model-provider conditions, six state-transfer protocols, two state budgets, and three runs. Dedicated final-answer success occurs on 696 rows (9.67%); intrinsic state validity on 191 (2.65%); answer-aligned validity on 180 (2.50%); and joint answer–carrier success on 173 (2.40%). Thus, only 24.86% of answer-correct outputs carry answer-aligned valid state, leaving a 75.14% silent-state-failure rate. In the three protocols with estimable valid and invalid strata, valid state is associated with higher next-step success, with task-cluster risk differences of 35.42–72.97 percentage points. Because the separately implemented executor shares task semantics with the validator, these contrasts provide criterion-related operational evidence rather than an unrelated external ground truth. Under this benchmark contract, final-answer accuracy is insufficient when downstream computation consumes the exposed state. All revised scores are replayed offline from saved outputs without model regeneration.

Authors

Institutions

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-09-12
DOI
https://doi.org/10.5281/zenodo.22720976
Primary Topic
Topic Modeling
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
preprint

When Correct Answers Carry Invalid State: Evaluating Explicit State Carriers in Stateful LLM Agents

Yuda Bi, Zhida Qin, Jiangwei Xue
Zenodo (CERN European Organization for Nuclear Research)
Topic Modeling
preprint

When Correct Answers Carry Invalid State: Evaluating Explicit State Carriers in Stateful LLM Agents

Yuda Bi, Zhida Qin, Jiangwei Xue
preprint en

Abstract

Stateful LLM agents often expose bounded state for later computation. A correct final answer does not establish that this state is safe, sufficient for continuation, or aligned with the answer. We operationalize this acceptance question with a task-relative, field-sensitive, representation-normalized contract for explicit state carriers. Deterministic adapters map native summary, visible-carry, and typed-state representations into a common claimed-state layer; the contract then evaluates parseability, actionable-state presence, exclusion safety, operational sufficiency, and answer–state consistency. We apply the contract to 7,200 frozen outputs over 40 tasks, five model-provider conditions, six state-transfer protocols, two state budgets, and three runs. Dedicated final-answer success occurs on 696 rows (9.67%); intrinsic state validity on 191 (2.65%); answer-aligned validity on 180 (2.50%); and joint answer–carrier success on 173 (2.40%). Thus, only 24.86% of answer-correct outputs carry answer-aligned valid state, leaving a 75.14% silent-state-failure rate. In the three protocols with estimable valid and invalid strata, valid state is associated with higher next-step success, with task-cluster risk differences of 35.42–72.97 percentage points. Because the separately implemented executor shares task semantics with the validator, these contrasts provide criterion-related operational evidence rather than an unrelated external ground truth. Under this benchmark contract, final-answer accuracy is insufficient when downstream computation consumes the exposed state. All revised scores are replayed offline from saved outputs without model regeneration.

Zenodo (CERN European Organization for Nuclear Research)
Center for Translational Research in Neuroimaging and Data Science (US)
Reduced inequalities
Topic Modeling
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.