Think and Answer Tokens Use Overlapping but Distinct Components of a Reasoning Model: Phase Restricted Ablations Against Token Group Baselines

Reasoning models write a chain of thought before a final answer. Do the think and answer phases recruit different internal components? We mean ablate every attention head, attention layer and MLP of DeepSeek-R1-Distill-Qwen-1.5B on its own GSM8K traces and compare the importance profiles of think and answer tokens against their own measurement noise, with a disattenuated agreement statistic, bootstrap intervals, matched nulls and planted controls. Ablating each component only at the positions of the phase being scored, instead of everywhere, lowers head level agreement from 0.80 to 0.73 (95% CI 0.70 to 0.76). To give such values a scale we apply the same estimator to other pairs of token groups. Early and late think tokens agree at 0.91, while hard and easy think tokens agree at only 0.77, so in this model the phases differ about as much as easy and hard tokens do; matching think tokens to the answer tokens' difficulty does not raise agreement. Over the most important heads agreement is much lower (0.62 for the top 34). The answer share of importance rises with depth, but easy think tokens show a similar rise. Residual swaps with a displacement matched control confirm late layer answer specificity and trace negative think side effects to sharpened predictions. A MATH subset replicates these results. On Qwen3-1.7B restricted ablation again lowers agreement (0.81) and the top heads agree even less (0.38), but there the phases differ more than hard and easy tokens do and the depth trend survives difficulty matching. Its residual swaps show a different pattern. Random head sets show that the lower agreement of whole layers need not reflect layer structure. The phases share a backbone and differ measurably, most among the components that matter.

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-11-16
DOI
https://doi.org/10.5281/zenodo.24271756
Primary Topic
Explainable Artificial Intelligence (XAI)
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Think and Answer Tokens Use Overlapping but Distinct Components of a Reasoning Model: Phase Restricted Ablations Against Token Group Baselines

Sharven Rane
Zenodo (CERN European Organization for Nuclear Research)
Explainable Artificial Intelligence (XAI)
preprint

Think and Answer Tokens Use Overlapping but Distinct Components of a Reasoning Model: Phase Restricted Ablations Against Token Group Baselines

Sharven Rane
preprint en

Abstract

Reasoning models write a chain of thought before a final answer. Do the think and answer phases recruit different internal components? We mean ablate every attention head, attention layer and MLP of DeepSeek-R1-Distill-Qwen-1.5B on its own GSM8K traces and compare the importance profiles of think and answer tokens against their own measurement noise, with a disattenuated agreement statistic, bootstrap intervals, matched nulls and planted controls. Ablating each component only at the positions of the phase being scored, instead of everywhere, lowers head level agreement from 0.80 to 0.73 (95% CI 0.70 to 0.76). To give such values a scale we apply the same estimator to other pairs of token groups. Early and late think tokens agree at 0.91, while hard and easy think tokens agree at only 0.77, so in this model the phases differ about as much as easy and hard tokens do; matching think tokens to the answer tokens' difficulty does not raise agreement. Over the most important heads agreement is much lower (0.62 for the top 34). The answer share of importance rises with depth, but easy think tokens show a similar rise. Residual swaps with a displacement matched control confirm late layer answer specificity and trace negative think side effects to sharpened predictions. A MATH subset replicates these results. On Qwen3-1.7B restricted ablation again lowers agreement (0.81) and the top heads agree even less (0.38), but there the phases differ more than hard and easy tokens do and the depth trend survives difficulty matching. Its residual swaps show a different pattern. Random head sets show that the lower agreement of whole layers need not reflect layer structure. The phases share a backbone and differ measurably, most among the components that matter.

Zenodo (CERN European Organization for Nuclear Research)
Explainable Artificial Intelligence (XAI)
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon