Modus_X 2.1.0: Coordinated Dual Memory and Matched Language Scaling

Modus_X 2.1.0 is an experimental research release on coordinating bounded associative matrix memory with recurrent vector computation. It separates two evidence leads: MemoryFeedbackArchive for byte-level language modeling and CurrentArchiveDelta for controlled versioned-memory evaluation. CurrentArchiveDelta maintains shared-address current and archive matrices with independently gated update timescales. MemoryFeedbackArchive compresses their mixed retrieval through a low-rank projection and gates it into the vector-stream input before recurrence. This specific coordination pattern is the architectural contribution; fast weights, delta-rule updates, recurrent memory, and hybrid sequence models have substantial prior art. At a fixed 102.4M-character enwik8 budget, MemoryFeedbackArchive improves strongly from 47.44M to 81.49M parameters, then saturates from 81.49M to 99.44M. After matched late annealing, the promoted 81.49M model reaches 1.382445 dense test BPC at 163.84M characters, narrowly improving Modus_X v1.1.1 with 1.54% fewer parameters. Official Mamba remains better on the matched dense endpoint, while MemoryFeedback remains better than the tested official xLSTM. In controlled equal-state experiments, CurrentArchiveDelta outperforms a truncated Transformer KV baseline when the cache cannot retain full context, while full-context KV remains superior. The language-modeling and controlled-memory advantages have not yet been demonstrated in one trained checkpoint. The 1.1 BPC target and trained 1B quality remain open. The archive includes source code, experimental protocols, dense evaluation evidence, controlled-memory studies, rejected interventions, systems-readiness appendices, a restructured whitepaper, model card, provenance and reproducibility metadata, and an integrity manifest.

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-07-26
DOI
https://doi.org/10.5281/zenodo.20443698
Primary Topic
Topic Modeling
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
preprint

Modus_X 2.1.0: Coordinated Dual Memory and Matched Language Scaling

Sanyam Chaudhary
Zenodo (CERN European Organization for Nuclear Research)
Topic Modeling
preprint

Modus_X 2.1.0: Coordinated Dual Memory and Matched Language Scaling

Sanyam Chaudhary
preprint en

Abstract

Modus_X 2.1.0 is an experimental research release on coordinating bounded associative matrix memory with recurrent vector computation. It separates two evidence leads: MemoryFeedbackArchive for byte-level language modeling and CurrentArchiveDelta for controlled versioned-memory evaluation. CurrentArchiveDelta maintains shared-address current and archive matrices with independently gated update timescales. MemoryFeedbackArchive compresses their mixed retrieval through a low-rank projection and gates it into the vector-stream input before recurrence. This specific coordination pattern is the architectural contribution; fast weights, delta-rule updates, recurrent memory, and hybrid sequence models have substantial prior art. At a fixed 102.4M-character enwik8 budget, MemoryFeedbackArchive improves strongly from 47.44M to 81.49M parameters, then saturates from 81.49M to 99.44M. After matched late annealing, the promoted 81.49M model reaches 1.382445 dense test BPC at 163.84M characters, narrowly improving Modus_X v1.1.1 with 1.54% fewer parameters. Official Mamba remains better on the matched dense endpoint, while MemoryFeedback remains better than the tested official xLSTM. In controlled equal-state experiments, CurrentArchiveDelta outperforms a truncated Transformer KV baseline when the cache cannot retain full context, while full-context KV remains superior. The language-modeling and controlled-memory advantages have not yet been demonstrated in one trained checkpoint. The 1.1 BPC target and trained 1B quality remain open. The archive includes source code, experimental protocols, dense evaluation evidence, controlled-memory studies, rejected interventions, systems-readiness appendices, a restructured whitepaper, model card, provenance and reproducibility metadata, and an integrity manifest.

Zenodo (CERN European Organization for Nuclear Research)
Topic Modeling
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.