WEAVE: Forward-Written Constant-Size Matrix Memory with Runtime Multi-Hop Composition

The long tail of world knowledge is what makes language-model pretraining expensive. Grammar and reasoning circuits are finite, but facts are not — and composing separately-learned facts demands data proportional to the transitive closure (O(N^k) for N atoms), because transformers do not compose what they were not trained to compose (the two-hop and reversal curses). WEAVE breaks this coupling by separating knowledge from skill. A constant-size D×D matrix memory S acquires facts through forward-only outer products — one pass, zero backpropagation — and composes them at inference time via S^k with a per-hop codebook snap. Weights learn only skill: grammar, extraction, and the walk. The novelty is (1) runtime S^k composition of forward-written facts, and (2) four capabilities — gradient-free write, runtime compose, targeted unlearn, and abstain — unified in one constant forward-written substrate. In a controlled head-to-head on identical single-hop facts (N<=1000), WEAVE achieves 100% zero-shot two- and three-hop composition where a transformer memorizes training hops perfectly but scores 0% on held-out two-hop queries (the two-hop curse). Built-in negative controls — shuffled linkage and direct cross-hop reads both scoring 0% — confirm genuine chain traversal. Reasoning depth decouples from network depth: S^k runs in zero layers while the language-model trunk knee is L4. One-shot recall reaches 37.9% versus 0.7% for gradient training. 2000 sequential read-erase-write edits accumulate no damage, where ROME/MEMIT collapses after dozens. An end-to-end conversational demonstration (49M-parameter trunk on Volta-16GB hardware, store logit injection) raises seeded-fact probability from <=0.08 to >=0.96 while unseeded entities remain at <=0.0002 — abstention, not hallucination. This is a preprint. Independent research.

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-08-27
DOI
https://doi.org/10.5281/zenodo.22126499
Primary Topic
Ferroelectric and Negative Capacitance Devices
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
preprint

WEAVE: Forward-Written Constant-Size Matrix Memory with Runtime Multi-Hop Composition

Jihun Kim
Zenodo (CERN European Organization for Nuclear Research)
Ferroelectric and Negative Capacitance Devices
preprint

WEAVE: Forward-Written Constant-Size Matrix Memory with Runtime Multi-Hop Composition

Jihun Kim
preprint en

Abstract

The long tail of world knowledge is what makes language-model pretraining expensive. Grammar and reasoning circuits are finite, but facts are not — and composing separately-learned facts demands data proportional to the transitive closure (O(N^k) for N atoms), because transformers do not compose what they were not trained to compose (the two-hop and reversal curses). WEAVE breaks this coupling by separating knowledge from skill. A constant-size D×D matrix memory S acquires facts through forward-only outer products — one pass, zero backpropagation — and composes them at inference time via S^k with a per-hop codebook snap. Weights learn only skill: grammar, extraction, and the walk. The novelty is (1) runtime S^k composition of forward-written facts, and (2) four capabilities — gradient-free write, runtime compose, targeted unlearn, and abstain — unified in one constant forward-written substrate. In a controlled head-to-head on identical single-hop facts (N<=1000), WEAVE achieves 100% zero-shot two- and three-hop composition where a transformer memorizes training hops perfectly but scores 0% on held-out two-hop queries (the two-hop curse). Built-in negative controls — shuffled linkage and direct cross-hop reads both scoring 0% — confirm genuine chain traversal. Reasoning depth decouples from network depth: S^k runs in zero layers while the language-model trunk knee is L4. One-shot recall reaches 37.9% versus 0.7% for gradient training. 2000 sequential read-erase-write edits accumulate no damage, where ROME/MEMIT collapses after dozens. An end-to-end conversational demonstration (49M-parameter trunk on Volta-16GB hardware, store logit injection) raises seeded-fact probability from <=0.08 to >=0.96 while unseeded entities remain at <=0.0002 — abstention, not hallucination. This is a preprint. Independent research.

Zenodo (CERN European Organization for Nuclear Research)
Quality Education
Ferroelectric and Negative Capacitance Devices
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.