The Sliding Window Premium: Throughput, Goodput, and Per Layer Attention Costs of Hybrid Attention Models at Realistic Context Lengths on TPU v6e

This paper is the fifth study in a series measuring the vLLM TPU serving stack, and the first to read hardware traces per attention layer type. Gemma 4 31B, a hybrid attention model with fifty sliding window layers and ten global layers, is served on a TPU v6e slice of four chips, and the decode attention cost of each layer type is measured from compiled kernel names across histories from 140 to 8,000 tokens. Three results organize the paper. First, the sliding window premium: at short histories, where windowed and global layers read identical data, the windowed layers cost 53.1 percent more per layer, a premium that survives the shipped heuristic, a targeted kernel intervention certified as a null at a noise floor of 0.16 percent, and a change of KV cache precision, and that never inverts when normalized per block of work, locating the mechanism in the per grid step cost of the sliding window kernel variant. Second, the crossover: the per layer ordering flips between 1,300 and 4,000 tokens, and at 8,000 tokens the ten global layers cost as much as the fifty windowed layers combined. Third, the mixed block size policy: the two layer types have opposite block size optima at long history, the shipped heuristic compiles 2,048 token blocks onto layers that can never attend past 1,024, and assigning each layer type its measured optimum computes to about 32 percent less decode attention at 8,000 tokens than either uniform configuration, implementable because the stack already compiles the layer types as separate kernels. The campaign also relocates the series' long context reversal boundary below 1,300 tokens, shows the goodput cliff moves with history at fixed batch, documents that every prior run labeled block size 16, 32, or 64 had silently compiled to 128, and corrects the series' determinism claim to statistical rather than bit determinism across runs. Economic and multi agent serving implications are quantified, and the complete measurement harness is released.