TUPOI: Attention-Free Post-Transformer with Constant O(1) Memory
While standard Transformer architectures (GPT, Claude) suffer from quadratic complexity O(N^2) during training and exploding Key-Value caches during generation, we introduce TUPOI — a foundational attention-free architecture that operates with strictly constant O(1) memory (64.00 KB) and multi-head Delta-KV associative memory. TUPOI represents sequence tokens as canonical position-momentum coordinates (q, p) governed by a 2nd-order Velocity-Verlet leapfrog integrator that strictly preserves Liouville phase volume (det(J) = 1.000000). On the Multi-Query Associative Recall (MQAR) benchmark, TUPOI achieves 100.0% accuracy (surpassing a matched causal transformer baseline at 24.4%) while achieving a 158.8x generation speedup at 32,768 context length. Official repository and code: https://github.com/starface77/TUPOI
Authors
- Danil Vladov
Institutions
- National Archives and Records Administration (US)
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-08-18
- DOI
- https://doi.org/10.5281/zenodo.22001817
- Primary Topic
- Parallel Computing and Optimization Techniques
- Type
- preprint