Told Once: One Memory Entry per Name in a Frozen Language Model
A language model that is told a fact after its training has two places to keep it: the prompt, or its weights. We describe a third. Some open models now carry a lookup memory indexed by hashed n-grams, whose retrieved vector is added to the residual stream through a learned gate. We add one entry per name to that memory, keyed by the name's exact tokens, and find its value from a fitted start by a few minimum-norm Gauss–Newton steps, each the least change that meets conditions built from the model's own replies about people it already knows. Linear reads at the end of the question admit the entry only for a kind of question the name was told. On 120 made-up people, each told a birthplace, a university and an occupation, with nothing in the prompt and its reasoning switched off, the model answers the told facts in the wordings its solve was given (360 of 360) and in five wordings it was not (587 of 600). It answers one-step consequences that were never written anywhere 481 of 600 (102 untouched); where a name's two facts point to different answers it is right for 147 of 225, and most of its misses answer from the wrong told fact. The reads keep the entry out of all 2,520 questions about things the name was not told, and no entry fires on 500 questions about real people, so those replies are the untouched model's, string for string. No trained weight changes: an entry adds 40,960 new weights to the model's memory, computed in about 50 seconds and kept in a file the model loads at start. We report the faults found on the way and what the method does not yet do: three kinds of fact, one step of consequence, and a first run on one of the field's benchmarks, where the two-hop question is answered for 57 of 151 edited entities (7 untouched) and the edited entity's other answers are disturbed.
Authors
- Jonathan Washburn (ORCID: https://orcid.org/0009-0001-8868-7497)
Publication Details
- Published
- 2026-11-14
- DOI
- https://doi.org/10.67347/pith.preprint.2026.000067
- Primary Topic
- Topic Modeling
- Type
- preprint