Distributed Engrammics: Gradient-Free Transfer of a Persistent Fast-Weight Associative State Between Agents

Distributed Engrammics, preprint and reproducible code. Can a skill be transferred from one AI instance to another without fine-tuning, without backpropagation, and without updating model weights? This work treats a model's fast-weight state as a persistent, transferable object, an "engram", that can be written, read, moved between instances, forgotten, and governed. Fast weight programmers treat the fast-weight matrix as an ephemeral memory, reset every sequence and private to one network. We study the opposite: this matrix F as a persistent, transferable object exchanged between agents, with four operations (inscribe, read, transfer, forget) and a subspace governance rule, tested through six pre-specified falsifiable hypotheses. In a controlled associative-memory backend (60 seeds, paired-bootstrap 95% CIs, Holm–Bonferroni), a low-rank state delta extracted from one agent transfers a skill to another without any gradient step, separating from both a no-transfer and a norm-matched random control by ≈88 points, with a rank dose-response, preserved host skill, projection-based forgetting, and subspace-gated consent. We then run the identical protocol on real DeltaNet language-model backends (1.3B and 2.7B, via flash-linear-attention). The primary claim holds: a gradient-free engram added to a recipient's recurrent state transfers the skill (+54 to +66 points over both controls across the two sizes, p < 0.001), with a monotone rank dose-response, and subspace governance succeeds once the authorized subspace is read from the model's actual keys (+62 points at 1.3B). Two idealized properties degrade: superposition interferes with the host skill, and subtraction-forgetting damages it. We trace this to shared key directions and confirm it by making the skills use disjoint symbols. Beyond memorized lookups, the fast state can carry a generalizing rule: a vowel/consonant classifier engram, injected into another instance, classifies held-out letters above no-transfer, random, and label-swapped controls on DeltaNet, and largely replicates on a second architecture (RWKV-7, where it clearly beats the no-transfer and label-specificity controls). On RWKV-7-2.9B, which can induce a novel rule in context, the engram of a freshly learned Caesar shift transfers that non-trivial induced rule: a neutral recipient applies it to held-out letters at 0.44 versus 0.005 (random) and 0.08 (wrong-shift), p < 0.0001; this extends to a small family of induced rules (further shifts and a reflection, each beating a wrong-rule control), though specificity does not hold in a new digit domain. Most experiments use two instances of one frozen checkpoint, where moving a state is close to state interpolation and the genuinely new content is the algebra of governance and forgetting; our most novel result lifts that restriction. Between two distinct RWKV-7 checkpoints, naive injection fails at chance while a learned linear key alignment recovers it to the recipient's own ceiling (0.04 → 0.97): a closed-form, skill-independent map fit once per pair with both models frozen, so the per-skill transfer stays gradient-free. We position the method against activation-space task and function vectors and state-soup interpolation, and release a single reproducible harness whose backends are judged by identical criteria. What this record contains: the paper (LaTeX source + compiled PDF), a single reproducible harness with two backends (a controlled associative-memory model and real linear-attention language models), per-seed CSVs, and complete raw run logs. Code and full reproduction instructions: https://github.com/JacquesGariepy/engrammics Scope: the mechanism applies to linear-attention models with a single additive recurrent state (DeltaNet, RWKV-7), not standard softmax (GPT-like) transformers; a follow-up paper may explore extending it to those architectures.

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-06-26
DOI
https://doi.org/10.5281/zenodo.20931467
Primary Topic
Ferroelectric and Negative Capacitance Devices
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
preprint

Distributed Engrammics: Gradient-Free Transfer of a Persistent Fast-Weight Associative State Between Agents

Jacques Gariepy
Zenodo (CERN European Organization for Nuclear Research)
Ferroelectric and Negative Capacitance Devices
preprint

Distributed Engrammics: Gradient-Free Transfer of a Persistent Fast-Weight Associative State Between Agents

Jacques Gariepy
preprint en

Abstract

Distributed Engrammics, preprint and reproducible code. Can a skill be transferred from one AI instance to another without fine-tuning, without backpropagation, and without updating model weights? This work treats a model's fast-weight state as a persistent, transferable object, an "engram", that can be written, read, moved between instances, forgotten, and governed. Fast weight programmers treat the fast-weight matrix as an ephemeral memory, reset every sequence and private to one network. We study the opposite: this matrix F as a persistent, transferable object exchanged between agents, with four operations (inscribe, read, transfer, forget) and a subspace governance rule, tested through six pre-specified falsifiable hypotheses. In a controlled associative-memory backend (60 seeds, paired-bootstrap 95% CIs, Holm–Bonferroni), a low-rank state delta extracted from one agent transfers a skill to another without any gradient step, separating from both a no-transfer and a norm-matched random control by ≈88 points, with a rank dose-response, preserved host skill, projection-based forgetting, and subspace-gated consent. We then run the identical protocol on real DeltaNet language-model backends (1.3B and 2.7B, via flash-linear-attention). The primary claim holds: a gradient-free engram added to a recipient's recurrent state transfers the skill (+54 to +66 points over both controls across the two sizes, p < 0.001), with a monotone rank dose-response, and subspace governance succeeds once the authorized subspace is read from the model's actual keys (+62 points at 1.3B). Two idealized properties degrade: superposition interferes with the host skill, and subtraction-forgetting damages it. We trace this to shared key directions and confirm it by making the skills use disjoint symbols. Beyond memorized lookups, the fast state can carry a generalizing rule: a vowel/consonant classifier engram, injected into another instance, classifies held-out letters above no-transfer, random, and label-swapped controls on DeltaNet, and largely replicates on a second architecture (RWKV-7, where it clearly beats the no-transfer and label-specificity controls). On RWKV-7-2.9B, which can induce a novel rule in context, the engram of a freshly learned Caesar shift transfers that non-trivial induced rule: a neutral recipient applies it to held-out letters at 0.44 versus 0.005 (random) and 0.08 (wrong-shift), p < 0.0001; this extends to a small family of induced rules (further shifts and a reflection, each beating a wrong-rule control), though specificity does not hold in a new digit domain. Most experiments use two instances of one frozen checkpoint, where moving a state is close to state interpolation and the genuinely new content is the algebra of governance and forgetting; our most novel result lifts that restriction. Between two distinct RWKV-7 checkpoints, naive injection fails at chance while a learned linear key alignment recovers it to the recipient's own ceiling (0.04 → 0.97): a closed-form, skill-independent map fit once per pair with both models frozen, so the per-skill transfer stays gradient-free. We position the method against activation-space task and function vectors and state-soup interpolation, and release a single reproducible harness whose backends are judged by identical criteria. What this record contains: the paper (LaTeX source + compiled PDF), a single reproducible harness with two backends (a controlled associative-memory model and real linear-attention language models), per-seed CSVs, and complete raw run logs. Code and full reproduction instructions: https://github.com/JacquesGariepy/engrammics Scope: the mechanism applies to linear-attention models with a single additive recurrent state (DeltaNet, RWKV-7), not standard softmax (GPT-like) transformers; a follow-up paper may explore extending it to those architectures.

Zenodo (CERN European Organization for Nuclear Research)
Quality Education
Ferroelectric and Negative Capacitance Devices
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.