Told Once: One Memory Entry per Name in a Frozen Language Model

A language model that is told a fact after its training has two places to keep it: the prompt, or its weights. We describe a third. Some open models now carry a lookup memory indexed by hashed n-grams, whose retrieved vector is added to the residual stream through a learned gate. We add one entry per name to that memory, keyed by the name's exact tokens, and find its value from a fitted start by a few minimum-norm Gauss–Newton steps, each the least change that meets conditions built from the model's own replies about people it already knows. Linear reads at the end of the question admit the entry only for a kind of question the name was told. On 120 made-up people, each told a birthplace, a university and an occupation, with nothing in the prompt and its reasoning switched off, the model answers the told facts in the wordings its solve was given (360 of 360) and in five wordings it was not (587 of 600). It answers one-step consequences that were never written anywhere 481 of 600 (102 untouched); where a name's two facts point to different answers it is right for 147 of 225, and most of its misses answer from the wrong told fact. The reads keep the entry out of all 2,520 questions about things the name was not told, and no entry fires on 500 questions about real people, so those replies are the untouched model's, string for string. No trained weight changes: an entry adds 40,960 new weights to the model's memory, computed in about 50 seconds and kept in a file the model loads at start. We report the faults found on the way and what the method does not yet do: three kinds of fact, one step of consequence, and a first run on one of the field's benchmarks, where the two-hop question is answered for 57 of 151 edited entities (7 untouched) and the edited entity's other answers are disturbed.

Authors

Publication Details

Published
2026-11-14
DOI
https://doi.org/10.67347/pith.preprint.2026.000067
Primary Topic
Topic Modeling
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Told Once: One Memory Entry per Name in a Frozen Language Model

Jonathan Washburn
Topic Modeling
preprint

Told Once: One Memory Entry per Name in a Frozen Language Model

Jonathan Washburn
preprint en

Abstract

A language model that is told a fact after its training has two places to keep it: the prompt, or its weights. We describe a third. Some open models now carry a lookup memory indexed by hashed n-grams, whose retrieved vector is added to the residual stream through a learned gate. We add one entry per name to that memory, keyed by the name's exact tokens, and find its value from a fitted start by a few minimum-norm Gauss–Newton steps, each the least change that meets conditions built from the model's own replies about people it already knows. Linear reads at the end of the question admit the entry only for a kind of question the name was told. On 120 made-up people, each told a birthplace, a university and an occupation, with nothing in the prompt and its reasoning switched off, the model answers the told facts in the wordings its solve was given (360 of 360) and in five wordings it was not (587 of 600). It answers one-step consequences that were never written anywhere 481 of 600 (102 untouched); where a name's two facts point to different answers it is right for 147 of 225, and most of its misses answer from the wrong told fact. The reads keep the entry out of all 2,520 questions about things the name was not told, and no entry fires on 500 questions about real people, so those replies are the untouched model's, string for string. No trained weight changes: an entry adds 40,960 new weights to the model's memory, computed in about 50 seconds and kept in a file the model loads at start. We report the faults found on the way and what the method does not yet do: three kinds of fact, one step of consequence, and a first run on one of the field's benchmarks, where the two-hop question is answered for 57 of 151 edited entities (7 untouched) and the edited entity's other answers are disturbed.

Topic Modeling
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon