LATVEC: Stabilizing Iterative Latent Workspaces for Neural Sequence Models

Iterative latent workspaces offer an alternative computational paradigm for maintaining, routing, and transforming structured relational information during neural sequence processing. However, repeatedly updating a shared continuous latent workspace across unconstrained recurrent iterations frequently leads to rapid representational collapse, wherein initially diverse slot representations contract into a degenerate low-rank subspace. We introduce **LATVEC**, an instrumented multi-slot latent workspace architecture designed to empirically investigate and govern this phenomenon. LATVEC combines a causal autoregressive Transformer perception backbone with an attention-free, gated pairwise communication workspace operating over $M=48$ discrete slots. We trace workspace stability via continuous effective-rank trajectories ($r_{\text{eff}}$) and systematically explore the interaction between recurrent update step sizes ($\alpha$) and off-diagonal gate sparsity regularization ($\lambda_{\text{gate}}$). Across rigorous benchmarks, unconstrained workspace dynamics exhibit rapid rank collapse ($r_{\text{eff}} \to 1.00$), whereas a jointly constrained regime ($\alpha \in [0.02, 0.05], \lambda_{\text{gate}} \in [0.01, 0.10]$) robustly stabilizes representation dimensionality ($r_{\text{eff}} \approx 8.45$ in small models, $r_{\text{eff}} \approx 4.93$ at 251.3M parameter scale). Multi-seed evaluations confirm this stability is reproducible across both pre-training and response-masked supervised fine-tuning (SFT). We then conduct extensive causal and functional ablations, evaluating active workspace dynamics against a zeroed workspace bypass (backbone-only control) and a norm-matched Gaussian scrambled negative control, alongside alternative readout interfaces (single-head vs. multi-head cross-attention). State randomization severely degrades task performance (3.1% accuracy), demonstrating that the learned workspace representation is functionally consumed by downstream decoding layers. However, under compute-matched conditions across 2 to 10-hop relational reasoning chains, the iterative workspace does not demonstrate a statistical advantage over the standard Transformer backbone control (35.7% vs. 39.3%). These empirical findings establish an essential conceptual distinction between **workspace stability** and **task-level superiority**: preventing recurrent rank collapse is a necessary engineering prerequisite for stable iterative computation, but does not inherently confer cognitive reasoning advantages over standard attention mechanisms. We present LATVEC as a foundational mechanistic study of latent-state dynamics, collapse prevention, and functional workspace utilization in neural sequence architectures.