Zero-Interference Associative Recall in a Single Linear Recurrent Language Model: Frozen Stable Keys, a Trigger Gate, and Decoupled Training

Autoregressive language modeling must do two things at once: compress a mostly-local context into a state that drives fluent next-token prediction, and, on occasion, re-encounter a previously seen key and emit the exact value bound to it. Linear-time recurrent and state-space models do the first at constant per-step cost but recall a far-away value poorly; attention recalls exactly but is quadratic. A common resolution attaches a separate memory or retrieval module. We challenge that partition with an attention-free linear backbone of Mamba2-style gated blocks and MLP blocks, into which we splice a frozen-key, trigger-gated associative readout addressed by stable token keys (orthogonal by default). The readout contributes nothing to the language-model stream: on held-out WikiText-103, perplexity measured on identical sequences with the readout on and off is identical (Δ = +0.0 on validation 48.37 and test 56.60), while multi-query associative recall is exact (1.000 at nearest, mid, and farthest distance) within the tested regime (store size ≤ 256, stable-key dimension 256, marker-token required). Against matched baselines a Mamba-2-style SSM, a no-forgetting linear-attention accumulator, and a full-attention Transformer all recover none of the stored pairs. We further test the necessity of the orthogonal key family: frozen random keys are equally exact at load one, so orthogonality is a sufficient default for near-key discrimination rather than a requirement, and we give the first direct measurements of the dk=256 model past load one (loads 2–4: nearest and mid recall remain exact, degradation concentrates on the oldest binding). This is a proof of principle at the load-one boundary with an explicit trigger signal, at a single model scale. Version 3 is a major revision of 10.5281/zenodo.22243420. The title and framing are changed to state the claim scope explicitly (zero-interference coexistence under frozen stable keys, a trigger gate, and decoupled training, at the load-one boundary). New experimental content in this version: (i) a controlled ablation showing that the orthogonal key family is a sufficient default rather than a necessary condition for exact recall at load one (frozen random keys are equally exact); and (ii) the first direct measurements of the dk=256 model past load one (loads 2–4), showing that nearest and mid-distance recall remain exact while degradation concentrates on the oldest binding. We also add per-seed robustness tables, a plain-language motivation, an expanded reference list, and precise statements of the tested regime (store size ≤ 256, stable-key dimension 256, marker token required, single model scale). This record continues the author's series on content-addressed recursive state: 10.5281/zenodo.22184105 (v1) and 10.5281/zenodo.22243420 (v2).

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-09-07
DOI
https://doi.org/10.5281/zenodo.22184104
Primary Topic
Handwritten Text Recognition Techniques
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
preprint

Zero-Interference Associative Recall in a Single Linear Recurrent Language Model: Frozen Stable Keys, a Trigger Gate, and Decoupled Training

Ziheng Zhou
Zenodo (CERN European Organization for Nuclear Research)
Handwritten Text Recognition Techniques
preprint

Zero-Interference Associative Recall in a Single Linear Recurrent Language Model: Frozen Stable Keys, a Trigger Gate, and Decoupled Training

Ziheng Zhou
preprint en

Abstract

Autoregressive language modeling must do two things at once: compress a mostly-local context into a state that drives fluent next-token prediction, and, on occasion, re-encounter a previously seen key and emit the exact value bound to it. Linear-time recurrent and state-space models do the first at constant per-step cost but recall a far-away value poorly; attention recalls exactly but is quadratic. A common resolution attaches a separate memory or retrieval module. We challenge that partition with an attention-free linear backbone of Mamba2-style gated blocks and MLP blocks, into which we splice a frozen-key, trigger-gated associative readout addressed by stable token keys (orthogonal by default). The readout contributes nothing to the language-model stream: on held-out WikiText-103, perplexity measured on identical sequences with the readout on and off is identical (Δ = +0.0 on validation 48.37 and test 56.60), while multi-query associative recall is exact (1.000 at nearest, mid, and farthest distance) within the tested regime (store size ≤ 256, stable-key dimension 256, marker-token required). Against matched baselines a Mamba-2-style SSM, a no-forgetting linear-attention accumulator, and a full-attention Transformer all recover none of the stored pairs. We further test the necessity of the orthogonal key family: frozen random keys are equally exact at load one, so orthogonality is a sufficient default for near-key discrimination rather than a requirement, and we give the first direct measurements of the dk=256 model past load one (loads 2–4: nearest and mid recall remain exact, degradation concentrates on the oldest binding). This is a proof of principle at the load-one boundary with an explicit trigger signal, at a single model scale. Version 3 is a major revision of 10.5281/zenodo.22243420. The title and framing are changed to state the claim scope explicitly (zero-interference coexistence under frozen stable keys, a trigger gate, and decoupled training, at the load-one boundary). New experimental content in this version: (i) a controlled ablation showing that the orthogonal key family is a sufficient default rather than a necessary condition for exact recall at load one (frozen random keys are equally exact); and (ii) the first direct measurements of the dk=256 model past load one (loads 2–4), showing that nearest and mid-distance recall remain exact while degradation concentrates on the oldest binding. We also add per-seed robustness tables, a plain-language motivation, an expanded reference list, and precise statements of the tested regime (store size ≤ 256, stable-key dimension 256, marker token required, single model scale). This record continues the author's series on content-addressed recursive state: 10.5281/zenodo.22184105 (v1) and 10.5281/zenodo.22243420 (v2).

Zenodo (CERN European Organization for Nuclear Research)
Handwritten Text Recognition Techniques
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.