Non-Lingering Activations: A Temporal Decay Compute Fabric for Transformer Inference

Intermediate activations in Transformer inference are consumed exactly once and never re-read. Their persistence in HBM is not a functional requirement; it is an architectural default. We propose a circuit fabric that eliminates this default by applying the Temporal Decay Inequality (Kang, 2026; doi:10.5281/zenodo.20471664) to the inter-sublayer activation boundary: the activation state is engineered to decay below the logic discrimination threshold before the HBM write path becomes circuit-topologically established, forwarding the result directly to the downstream sublayer and producing no HBM artifact. We derive the traffic reduction analytically for LLaMA-3 70B, anchored to an HBM3 memory subsystem of H100-equivalent bandwidth (3.35 TB/s). At batch size 32, sequence length 512, cross-sublayer activation traffic is 322 GB per forward pass against a weight traffic baseline of 137 GB. The non-lingering activation fabric eliminates the activation component entirely, reducing total HBM traffic by 70%, with an activation HBM transaction time of 96 ms per forward pass at peak bandwidth — an upper bound on latency recoverable from this component. At batch size 1, sequence length 4096, the reduction is 37%. The benefit scales with the product B·S of batch size and sequence length: at B=32, S=2048, reduction reaches 90%; at B=64, S=2048, 95%. Activation traffic grows with B·S while weight traffic does not, so the fabric's proportional advantage increases under the large-batch, long-context conditions that characterize production serving. The decay is spontaneous — no active erasure, no software intervention, no compression artifact. Functional correctness is preserved by the left inequality τ_compute < τ_state, which guarantees the activation reaches its consumer before decay. Two physical mechanisms within existing semiconductor technology — capacitance-leakage discharge and thermodynamic barrier crossing — provide continuous design control over the state lifetime required. This work is an application of the Temporal Decay Inequality, the foundational postulate for non-persistent computation introduced in the companion preprint (doi:10.5281/zenodo.20471664)