Removing the Competitor Is Not Enough: Reallocating Attention to the Current Value Restores Recall under Proactive Interference

Under proactive interference, language models answer with a value that has already been overwritten instead of a repeatedly updated key’s current one, well before the context window fills (Wang and Sun, 2025). For this source of competition, cumulative overwrite of one key, we ask whether an intervention that reduces these errors recovers the key’s binding or merely redirects the error, on Qwen2.5-1.5B-Instruct and two further families. On the main model, patching the clean activations of ten attention heads restores 54.1% of the gap on held-out items, and this localisation replicates in one of the two further families, while the residual-direction account is not supported (r = −0.006). With the same attention cells masked in every arm, reallocating the answer position’s attention from the overwritten values to the queried key’s current value beats sending the same mass elsewhere by +0.63, +0.49 and +0.39 across the three families, intervals excluding zero, and restores clean accuracy when applied at every head, whereas removal alone does not; a wrong-address control in Phi-3.5-mini shows that the destination, not the quantity of attention, decides recovery. The destination is supplied by an oracle, and the same redirection gains 13 to 20 points with no competitor present: under proactive interference what fails is where the answer position looks, and the evidence licenses recovery given the right address, not a binding that was intact and merely unread.

Authors

Institutions

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-09-25
DOI
https://doi.org/10.5281/zenodo.22961473
Primary Topic
Topic Modeling
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
preprint

Removing the Competitor Is Not Enough: Reallocating Attention to the Current Value Restores Recall under Proactive Interference

Ya-Fen Yeh, Guan-Yuan Chen
Zenodo (CERN European Organization for Nuclear Research)
Topic Modeling
preprint

Removing the Competitor Is Not Enough: Reallocating Attention to the Current Value Restores Recall under Proactive Interference

Ya-Fen Yeh, Guan-Yuan Chen
preprint en

Abstract

Under proactive interference, language models answer with a value that has already been overwritten instead of a repeatedly updated key’s current one, well before the context window fills (Wang and Sun, 2025). For this source of competition, cumulative overwrite of one key, we ask whether an intervention that reduces these errors recovers the key’s binding or merely redirects the error, on Qwen2.5-1.5B-Instruct and two further families. On the main model, patching the clean activations of ten attention heads restores 54.1% of the gap on held-out items, and this localisation replicates in one of the two further families, while the residual-direction account is not supported (r = −0.006). With the same attention cells masked in every arm, reallocating the answer position’s attention from the overwritten values to the queried key’s current value beats sending the same mass elsewhere by +0.63, +0.49 and +0.39 across the three families, intervals excluding zero, and restores clean accuracy when applied at every head, whereas removal alone does not; a wrong-address control in Phi-3.5-mini shows that the destination, not the quantity of attention, decides recovery. The destination is supplied by an oracle, and the same redirection gains 13 to 20 points with no competitor present: under proactive interference what fails is where the answer position looks, and the evidence licenses recovery given the right address, not a binding that was intact and merely unread.

Zenodo (CERN European Organization for Nuclear Research)
National Tsing Hua University (TW), North Carolina Exploring Cultural Heritage Online (US)
Topic Modeling
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Removing the Competitor Is Not Enough: Reallocating Attention to the Current Value Restores Recall under Proactive Interference — Ya-Fen Yeh, Guan-Yuan Chen · Zenodo (CERN European Organization for Nuclear Research) (2026) | TGRS Research Map | TGRS