Modus_X: Constant-State Language Modeling with Associative Memory and Selective Recurrence
Modus_X v1.1.1 is an open, attention-free causal sequence-modeling research release. The architecture combines two fixed-size recurrent mechanisms inside each layer: a selective vector recurrence for local sequential dynamics and a delta-rule associative matrix-memory stream for content-addressed retrieval and overwrite. Its inference state does not grow with generated sequence length in the way a Transformer KV cache does. This constant-state property is a systems direction, not by itself a claim of lower total compute in every deployment regime. This version adds a reproducible three-seed router and component ablation on a balanced key-value associative-recall protocol with both no-overwrite and 50% same-key-overwrite conditions. At length 2048, MatrixOnly retains 96.992 +/- 0.427% recall without overwrite and 87.625 +/- 0.745% with overwrite, while VectorOnly remains near the 3.125% chance level. The bounded conclusion is that the matrix-memory stream carries the tested associative binding and overwrite capability. The release includes the 24 raw seed-level result files, aggregate summaries, exact runners, figures, and reproduction instructions. The release also reports counterevidence transparently. On the matched 80M-tier enwik8 dense evaluation, the tested official Mamba configuration reaches 1.34578 BPC while Modus_X reaches 1.38418 BPC, so Modus_X does not yet beat Mamba on this byte-level compression benchmark. Modus_X exceeds the tested official xLSTM configuration, which reaches 1.41962 BPC under the same dense audit. These results support capability specialization rather than universal superiority: Modus_X shows strong controlled associative-memory behavior and constant recurrent state, while further scaling and natural-language evaluation remain necessary. Code, evidence, release archive, and the full whitepaper: https://github.com/sanyamChaudhary27/Modus_X
Authors
- Sanyam Chaudhary
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-06-26
- DOI
- https://doi.org/10.5281/zenodo.20923248
- Primary Topic
- Software System Performance and Reliability
- Type
- preprint