Fold, Don't Stream: Recurrent Depth as a Storage-Residency Optimization for Consumer-Device LLM Inference

When a language model exceeds a consumer device's RAM, decoding falls off a storage cliff: every token forces the active parameters to be re-read from disk. We measure it directly on an Apple M1 Max (32 GB), where a dense 70B model at 4-bit reads 40–44 GB from SSD per generated token and decodes at ∼0.02 tokens/s. Recurrent depth — executing a small stack of layers R times — is an established mechanism; we treat it as a lever against that cliff rather than as a parameter- or reasoning-efficiency one, and measure both what it buys and what it cannot. Controlled studies at 15M and 42M parameters show that at equal resident weights and equal training tokens a looped transformer recovers 41% (dense) and up to 54% (Mixture-of-Experts) of the quality gained by doubling memory, reproduced on two hardware/precision stacks. Inside looped MoE, sticky routing — re-using the first pass's expert selection — matches free re-routing within seed noise while freezing expert traffic at top-k, so access locality is free. A graph-folding patch to llama.cpp then removes the cliff on the same laptop: ∼120× for the dense 70B and ∼1.7× for a 118B-A8B MoE, with zero swap. Scoping the loader to the folded working set puts that 118B checkpoint on the GPU, where 57.6 GB of weights decode at 34.4 tokens/s with 25.2 GB wired — with degraded output. We then bound that degradation. Targeted repairs take the folded 118B from perplexity 4.2×10⁴ to 27.6 against 6.40 unfolded, and no further: a per-token least-squares solve over the physical layer's entire 256-expert set — a lower bound on any routing, router refit, or gating adapter — leaves 0.93–0.95 of the target residual, against a chance floor of 0.957 that any 256 directions in 3072 dimensions achieve. On the trajectory the fold produces, the physical layer's expert basis is barely more useful for its neighbour's routed function than a random subspace, so no gating-side repair closes the gap and quality must come from loop-native training. As a controlled endpoint, a flattened twin of a production looped model — function and output identical by construction — isolates residency as the sole cause of a ∼150× speed gap.