Voxel-Steered Autoregressive Generation: Overcoming the Quadratic Output Wall

Autoregressive language models face a strict quadratic wall: generation costs O(N²) in attention and O(N) in Key-Value (KV) cache memory. We present Voxel-Steered Autoregressive Generation, a geometric inversion of WP-20's spatial index. By implementing a “Rolling Output Eviction” loop, the model evicts recently generated chunks from the dense KV cache into a persistent HNSW index over the TERA cube, querying its own past geometry to maintain coherence. Furthermore, by predicting coarse semantic cells (Meji/Odu) before decoding discrete tokens, the model generates a trajectory through meaning-space rather than a sequence of characters. We demonstrate that this architecture allows infinite-length coherent generation with constant O(1) attention overhead and significant deduplication via Greedy Meshing. WP-21 extends WP-20 (Voxel-Addressable Memory) by inverting the retrieval mechanism from the input context space to the generative output space.

Authors

Institutions

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-06-27
DOI
https://doi.org/10.5281/zenodo.20945234
Primary Topic
Parallel Computing and Optimization Techniques
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

Voxel-Steered Autoregressive Generation: Overcoming the Quadratic Output Wall

Weslyn Cory Whitehead
Zenodo (CERN European Organization for Nuclear Research)
Parallel Computing and Optimization Techniques
article

Voxel-Steered Autoregressive Generation: Overcoming the Quadratic Output Wall

Weslyn Cory Whitehead
article en

Abstract

Autoregressive language models face a strict quadratic wall: generation costs O(N²) in attention and O(N) in Key-Value (KV) cache memory. We present Voxel-Steered Autoregressive Generation, a geometric inversion of WP-20's spatial index. By implementing a “Rolling Output Eviction” loop, the model evicts recently generated chunks from the dense KV cache into a persistent HNSW index over the TERA cube, querying its own past geometry to maintain coherence. Furthermore, by predicting coarse semantic cells (Meji/Odu) before decoding discrete tokens, the model generates a trajectory through meaning-space rather than a sequence of characters. We demonstrate that this architecture allows infinite-length coherent generation with constant O(1) attention overhead and significant deduplication via Greedy Meshing. WP-21 extends WP-20 (Voxel-Addressable Memory) by inverting the retrieval mechanism from the input context space to the generative output space.

Zenodo (CERN European Organization for Nuclear Research)
Klamath Community College (US)
Openalex Percentile: Top 4%
Parallel Computing and Optimization Techniques
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.