Language Models Do Not Re-Verify Computed Values in Their Own Reasoning

Language models exhibit increasingly remarkable capabilities on reasoning tasks, but it is unclear whether they spontaneously check values that are already part of their reasoning. In particular, we inject a single numerical error into model-generated execution traces for short mathematical programs across 14 language models from 5 developers, and study whether the models detect and correct the error or continue reasoning from it. Errors that require recomputation propagate at much higher rates than errors that can be caught by comparison alone. When the planted value explicitly contradicts a value written earlier in the trace, detection and correction rise up to 96-99% for stronger models. In contrast, when detecting the error requires even one arithmetic operation, unprompted continuation by the base models instead propagates the error in 88–100% of continuations, despite the models being able to compute the correct value when asked directly. This failure persists on most models even when they are given verification instructions and have both the information and computational ability needed to check the value, and it is not repaired by the activation-steering interventions we test. In contrast, fine-tuning on a few hundred planted-error examples are enough to teach some models to recheck previously computed values, and this behavior generalizes to harder errors than those seen during training. Our results expose a fragility in downstream self-correction even in highly capable models, and challenge the assumption that an incorrect intermediate computation will have a meaningful chance of being independently rechecked later in the reasoning process.

Authors

Publication Details

Journal
Stanford Digital Repository
Published
2026-11-19
DOI
https://doi.org/10.25740/qv877dr5692
Primary Topic
Topic Modeling
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Language Models Do Not Re-Verify Computed Values in Their Own Reasoning

Brando Miranda, Rylan Schaeffer, Sanmi Koyejo, Elyas Obbad et al.
Stanford Digital Repository
Topic Modeling
article

Language Models Do Not Re-Verify Computed Values in Their Own Reasoning

Brando Miranda, Rylan Schaeffer, Sanmi Koyejo, Elyas Obbad, Carmen Amo Alonso
article en

Abstract

Language models exhibit increasingly remarkable capabilities on reasoning tasks, but it is unclear whether they spontaneously check values that are already part of their reasoning. In particular, we inject a single numerical error into model-generated execution traces for short mathematical programs across 14 language models from 5 developers, and study whether the models detect and correct the error or continue reasoning from it. Errors that require recomputation propagate at much higher rates than errors that can be caught by comparison alone. When the planted value explicitly contradicts a value written earlier in the trace, detection and correction rise up to 96-99% for stronger models. In contrast, when detecting the error requires even one arithmetic operation, unprompted continuation by the base models instead propagates the error in 88–100% of continuations, despite the models being able to compute the correct value when asked directly. This failure persists on most models even when they are given verification instructions and have both the information and computational ability needed to check the value, and it is not repaired by the activation-steering interventions we test. In contrast, fine-tuning on a few hundred planted-error examples are enough to teach some models to recheck previously computed values, and this behavior generalizes to harder errors than those seen during training. Our results expose a fragility in downstream self-correction even in highly capable models, and challenge the assumption that an incorrect intermediate computation will have a meaningful chance of being independently rechecked later in the reasoning process.

Stanford Digital Repository
Topic Modeling
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon