Language Models Do Not Re-Verify Computed Values in Their Own Reasoning
Language models exhibit increasingly remarkable capabilities on reasoning tasks, but it is unclear whether they spontaneously check values that are already part of their reasoning. In particular, we inject a single numerical error into model-generated execution traces for short mathematical programs across 14 language models from 5 developers, and study whether the models detect and correct the error or continue reasoning from it. Errors that require recomputation propagate at much higher rates than errors that can be caught by comparison alone. When the planted value explicitly contradicts a value written earlier in the trace, detection and correction rise up to 96-99% for stronger models. In contrast, when detecting the error requires even one arithmetic operation, unprompted continuation by the base models instead propagates the error in 88–100% of continuations, despite the models being able to compute the correct value when asked directly. This failure persists on most models even when they are given verification instructions and have both the information and computational ability needed to check the value, and it is not repaired by the activation-steering interventions we test. In contrast, fine-tuning on a few hundred planted-error examples are enough to teach some models to recheck previously computed values, and this behavior generalizes to harder errors than those seen during training. Our results expose a fragility in downstream self-correction even in highly capable models, and challenge the assumption that an incorrect intermediate computation will have a meaningful chance of being independently rechecked later in the reasoning process.
Authors
- Brando Miranda (ORCID: https://orcid.org/0009-0008-5031-0126)
- Rylan Schaeffer (ORCID: https://orcid.org/0000-0002-4298-7216)
- Sanmi Koyejo (ORCID: https://orcid.org/0000-0002-4023-419X)
- Elyas Obbad
- Carmen Amo Alonso
Publication Details
- Journal
- Stanford Digital Repository
- Published
- 2026-11-19
- DOI
- https://doi.org/10.25740/qv877dr5692
- Primary Topic
- Topic Modeling
- Type
- article
- Field-Weighted Citation Impact
- 0.00