Origin-Paged Memory: Treating Language-Model Context as Virtual Memory A Pre-Registered First Test of the Origin-Paged Attention Program on a 4B-Parameter Model

Language-model agents that work for hours, or read documents far longer than their context window, need a principled way to decide what stays in context. We study Origin-Paged Attention (OPA), a research program that treats context like virtual memory in an operating system. The archive is split into fixed-size chunks called origins. The program has two layers. Its memory layer, Origin-Paged Memory (OPM), keeps a few origins in working slots, pins exact evidence, and pages an earlier origin back in when an entity becomes relevant late, using an index of the entities each origin mentions. Its attention layer, Origin-Anchored Attention (OAA), would move the same idea inside the model by giving every origin its own positional frame and an address in hyperbolic space. We prove that OAA attention factors exactly into a choice of origin times ordinary attention within it, so the error caused by paging equals the attention mass that was paged out. We then test 13 pre-registered hypotheses on the open Qwen3-4B-Instruct model, with every verdict rule fixed before any test data existed. On synthetic late-binding multi-hop questions at 32K tokens, OPM answered 76.6% correctly, against 62.4% for one-shot retrieval (a gain of 14.2 points, 95% CI 9.2–19.1); it also beat a gist-memory agent by 39.6 points, and pinning cut repeated page-ins by 2.54 per item. A graph of links between origins, hyperbolic distances, and a “red-dot” prefix that carries the end of one origin into the next added nothing measurable. At the attention level, a short fine-tuning run did not teach OAA to attribute facts to their origin (31.2% against 99.6% for standard attention), so the remaining attention-level and systems tests were not run. The memory layer is supported; moving it inside attention remains open.

Authors

Publication Details

Journal
Preprints.org
Published
2026-11-13
DOI
https://doi.org/10.20944/preprints202611.1367.v1
Primary Topic
Topic Modeling
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Origin-Paged Memory: Treating Language-Model Context as Virtual Memory A Pre-Registered First Test of the Origin-Paged Attention Program on a 4B-Parameter Model

Aaditya Rajgor
1 links
Preprints.org
Topic Modeling
preprint

Origin-Paged Memory: Treating Language-Model Context as Virtual Memory A Pre-Registered First Test of the Origin-Paged Attention Program on a 4B-Parameter Model

Aaditya Rajgor
preprint en

Abstract

Language-model agents that work for hours, or read documents far longer than their context window, need a principled way to decide what stays in context. We study Origin-Paged Attention (OPA), a research program that treats context like virtual memory in an operating system. The archive is split into fixed-size chunks called origins. The program has two layers. Its memory layer, Origin-Paged Memory (OPM), keeps a few origins in working slots, pins exact evidence, and pages an earlier origin back in when an entity becomes relevant late, using an index of the entities each origin mentions. Its attention layer, Origin-Anchored Attention (OAA), would move the same idea inside the model by giving every origin its own positional frame and an address in hyperbolic space. We prove that OAA attention factors exactly into a choice of origin times ordinary attention within it, so the error caused by paging equals the attention mass that was paged out. We then test 13 pre-registered hypotheses on the open Qwen3-4B-Instruct model, with every verdict rule fixed before any test data existed. On synthetic late-binding multi-hop questions at 32K tokens, OPM answered 76.6% correctly, against 62.4% for one-shot retrieval (a gain of 14.2 points, 95% CI 9.2–19.1); it also beat a gist-memory agent by 39.6 points, and pinning cut repeated page-ins by 2.54 per item. A graph of links between origins, hyperbolic distances, and a “red-dot” prefix that carries the end of one origin into the next added nothing measurable. At the attention level, a short fine-tuning run did not teach OAA to attribute facts to their origin (31.2% against 99.6% for standard attention), so the remaining attention-level and systems tests were not run. The memory layer is supported; moving it inside attention remains open.

Preprints.org
Topic Modeling
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon