Removing the Competitor Is Not Enough: Reallocating Attention to the Current Value Restores Recall under Proactive Interference
Under proactive interference, language models answer with a value that has already been overwritten instead of a repeatedly updated key’s current one, well before the context window fills (Wang and Sun, 2025). For this source of competition, cumulative overwrite of one key, we ask whether an intervention that reduces these errors recovers the key’s binding or merely redirects the error, on Qwen2.5-1.5B-Instruct and two further families. On the main model, patching the clean activations of ten attention heads restores 54.1% of the gap on held-out items, and this localisation replicates in one of the two further families, while the residual-direction account is not supported (r = −0.006). With the same attention cells masked in every arm, reallocating the answer position’s attention from the overwritten values to the queried key’s current value beats sending the same mass elsewhere by +0.63, +0.49 and +0.39 across the three families, intervals excluding zero, and restores clean accuracy when applied at every head, whereas removal alone does not; a wrong-address control in Phi-3.5-mini shows that the destination, not the quantity of attention, decides recovery. The destination is supplied by an oracle, and the same redirection gains 13 to 20 points with no competitor present: under proactive interference what fails is where the answer position looks, and the evidence licenses recovery given the right address, not a binding that was intact and merely unread.
Authors
- Ya-Fen Yeh
- Guan-Yuan Chen (ORCID: https://orcid.org/0000-0003-3298-0624)
Institutions
- National Tsing Hua University (TW)
- North Carolina Exploring Cultural Heritage Online (US)
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-09-25
- DOI
- https://doi.org/10.5281/zenodo.22961473
- Primary Topic
- Topic Modeling
- Type
- preprint