Modus_X: Constant-State Language Modeling with Associative Memory and Selective Recurrence

Modus_X v1.1.1 is an open, attention-free causal sequence-modeling research release. The architecture combines two fixed-size recurrent mechanisms inside each layer: a selective vector recurrence for local sequential dynamics and a delta-rule associative matrix-memory stream for content-addressed retrieval and overwrite. Its inference state does not grow with generated sequence length in the way a Transformer KV cache does. This constant-state property is a systems direction, not by itself a claim of lower total compute in every deployment regime. This version adds a reproducible three-seed router and component ablation on a balanced key-value associative-recall protocol with both no-overwrite and 50% same-key-overwrite conditions. At length 2048, MatrixOnly retains 96.992 +/- 0.427% recall without overwrite and 87.625 +/- 0.745% with overwrite, while VectorOnly remains near the 3.125% chance level. The bounded conclusion is that the matrix-memory stream carries the tested associative binding and overwrite capability. The release includes the 24 raw seed-level result files, aggregate summaries, exact runners, figures, and reproduction instructions. The release also reports counterevidence transparently. On the matched 80M-tier enwik8 dense evaluation, the tested official Mamba configuration reaches 1.34578 BPC while Modus_X reaches 1.38418 BPC, so Modus_X does not yet beat Mamba on this byte-level compression benchmark. Modus_X exceeds the tested official xLSTM configuration, which reaches 1.41962 BPC under the same dense audit. These results support capability specialization rather than universal superiority: Modus_X shows strong controlled associative-memory behavior and constant recurrent state, while further scaling and natural-language evaluation remain necessary. Code, evidence, release archive, and the full whitepaper: https://github.com/sanyamChaudhary27/Modus_X

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-06-26
DOI
https://doi.org/10.5281/zenodo.20923248
Primary Topic
Software System Performance and Reliability
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
preprint

Modus_X: Constant-State Language Modeling with Associative Memory and Selective Recurrence

Sanyam Chaudhary
Zenodo (CERN European Organization for Nuclear Research)
Software System Performance and Reliability
preprint

Modus_X: Constant-State Language Modeling with Associative Memory and Selective Recurrence

Sanyam Chaudhary
preprint en

Abstract

Modus_X v1.1.1 is an open, attention-free causal sequence-modeling research release. The architecture combines two fixed-size recurrent mechanisms inside each layer: a selective vector recurrence for local sequential dynamics and a delta-rule associative matrix-memory stream for content-addressed retrieval and overwrite. Its inference state does not grow with generated sequence length in the way a Transformer KV cache does. This constant-state property is a systems direction, not by itself a claim of lower total compute in every deployment regime. This version adds a reproducible three-seed router and component ablation on a balanced key-value associative-recall protocol with both no-overwrite and 50% same-key-overwrite conditions. At length 2048, MatrixOnly retains 96.992 +/- 0.427% recall without overwrite and 87.625 +/- 0.745% with overwrite, while VectorOnly remains near the 3.125% chance level. The bounded conclusion is that the matrix-memory stream carries the tested associative binding and overwrite capability. The release includes the 24 raw seed-level result files, aggregate summaries, exact runners, figures, and reproduction instructions. The release also reports counterevidence transparently. On the matched 80M-tier enwik8 dense evaluation, the tested official Mamba configuration reaches 1.34578 BPC while Modus_X reaches 1.38418 BPC, so Modus_X does not yet beat Mamba on this byte-level compression benchmark. Modus_X exceeds the tested official xLSTM configuration, which reaches 1.41962 BPC under the same dense audit. These results support capability specialization rather than universal superiority: Modus_X shows strong controlled associative-memory behavior and constant recurrent state, while further scaling and natural-language evaluation remain necessary. Code, evidence, release archive, and the full whitepaper: https://github.com/sanyamChaudhary27/Modus_X

Zenodo (CERN European Organization for Nuclear Research)
Software System Performance and Reliability
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.