Arithmetic Without Approximation: An Exact-Intervention Account of Number Representations in Open Language Models

How does a transformer compute a+b? An exact-intervention mechanistic account on three open checkpoints (Llama-3.1-8B, Pythia-6.9B, GPT-J-6B), scoring every intervention on the model's own emitted digit. The answer position carries two orthogonal number-derived subspaces with dissociated causal roles: X_contextual, the answer read-out, along which a write steers the emitted digit; and X_number, which carries the operands and is settable while an operand is being read but read-only once it is consumed, necessary to the computation without being where the answer is set. Two tidy pictures fail: representation is not operation, and no single helix carries the computation. Four measurement choices, the causal object, the null, the dependent-variable form, and the measurement layer, each flip a conclusion, and the same discipline exposes an apparent fourth-model result as an artifact. The manuscript PDF is a separate file on this record. The attached reproducibility archive contains all producer and figure scripts, the per-cell run data, a claims ledger mapping every headline number to its producing script (including all retractions), a run-JSON schema, tested dependency versions, and a checksum manifest. Code is licensed MIT; text and figures CC-BY-4.0.

Authors

Institutions

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-09-01
DOI
https://doi.org/10.5281/zenodo.22049437
Primary Topic
Mathematics, Computing, and Information Processing
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
preprint

Arithmetic Without Approximation: An Exact-Intervention Account of Number Representations in Open Language Models

Rob Cobbins
Zenodo (CERN European Organization for Nuclear Research)
Mathematics, Computing, and Information Processing
preprint

Arithmetic Without Approximation: An Exact-Intervention Account of Number Representations in Open Language Models

Rob Cobbins
preprint en

Abstract

How does a transformer compute a+b? An exact-intervention mechanistic account on three open checkpoints (Llama-3.1-8B, Pythia-6.9B, GPT-J-6B), scoring every intervention on the model's own emitted digit. The answer position carries two orthogonal number-derived subspaces with dissociated causal roles: X_contextual, the answer read-out, along which a write steers the emitted digit; and X_number, which carries the operands and is settable while an operand is being read but read-only once it is consumed, necessary to the computation without being where the answer is set. Two tidy pictures fail: representation is not operation, and no single helix carries the computation. Four measurement choices, the causal object, the null, the dependent-variable form, and the measurement layer, each flip a conclusion, and the same discipline exposes an apparent fourth-model result as an artifact. The manuscript PDF is a separate file on this record. The attached reproducibility archive contains all producer and figure scripts, the per-cell run data, a claims ledger mapping every headline number to its producing script (including all retractions), a run-JSON schema, tested dependency versions, and a checksum manifest. Code is licensed MIT; text and figures CC-BY-4.0.

Zenodo (CERN European Organization for Nuclear Research)
NeoPhotonics (United States) (US)
Quality Education
Mathematics, Computing, and Information Processing
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.