Modus_X 2.0.0: Coordinating Bounded Associative Memory with Recurrent Computation
Modus_X 2.0.0 is an experimental research release on coordinating bounded associative matrix memory with recurrent vector computation. It promotes two configurations for different evidence layers: MemoryFeedbackArchive for byte-level language modeling and CurrentArchiveDelta for controlled versioned-memory evaluation. CurrentArchiveDelta maintains shared-address current and archive matrices with independently gated update timescales. MemoryFeedbackArchive compresses their mixed retrieval through a low-rank projection and gates it into the vector-stream input before recurrence. This specific coordination pattern is the architectural contribution; fast weights, delta-rule updates, recurrent memory, and hybrid sequence models have substantial prior art. At a fixed 102.4M-character enwik8 budget, MemoryFeedbackArchive improves strongly from 47.44M to 81.49M parameters, then saturates from 81.49M to 99.44M. In controlled equal-state experiments, CurrentArchiveDelta outperforms a truncated Transformer KV baseline when the cache cannot retain full context, while full-context KV remains superior. Official Mamba remains better on the available matched dense enwik8 endpoint. The 1.1 BPC target and trained 1B quality remain open. The archive includes source, protocols, dense evaluation evidence, controlled-memory studies, rejected interventions, systems-readiness appendices, a full whitepaper, model card, provenance, reproducibility metadata, and an integrity manifest.
Authors
- Sanyam Chaudhary
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-07-25
- DOI
- https://doi.org/10.5281/zenodo.21538210
- Primary Topic
- Machine Learning in Materials Science
- Type
- preprint