WRAI: From Selective Routing to Fixed-State Memory for Autoregressive Language Models

This paper presents an integrated synthesis of the WRAI research program (Wavelet / Recurrent / Attentive Intelligence) from its initial selective-computation experiments to a family of fixed-state memory mechanisms for autoregressive inference. The research began by testing whether Transformer computation could be reduced through depth-aware layer routing, then shifted toward representing past context without retaining a token-indexed historical key-value (KV) cache. The resulting architecture combines positive feature maps, dynamically selected low-rank landmarks, recurrent numerator/normalizer states, depth-dependent feature temperature and decay, local Haar multi-resolution memory, attention-sink anchors, and salience-gated hyperdimensional computing (HDC) memory. On Qwen3-0.6B, the record reports progression from layer-local autoregressive replacement to a 28-layer standalone Tri-Memory path with zero historical Transformer KV writes under the tested execution path. A parallel transfer line evaluates Full-WRAI on the 35 text-attention positions of Gemma 4 E2B. Furthermore, BananaMind-2-Micro demonstrates fixed-state autoregressive execution directly on physical ESP32-D0WD-V3 silicon without external PSRAM in strictly constant O(1) memory. All accompanying models, telemetry logs, firmware, and reproduction artifacts are openly available at https://github.com/MusounoEnma/WRAI.