StairCut: sizing speculative draft trees on a measured verification-cost staircase

Decoding a dense 27-billion-parameter model on a unified-memory GPU is limited by the bytes each step reads, and speculative decoding is harder for hybrid models, because a recurrent state cannot be rolled back. This paper presents TandemLLM, an inference engine for Qwen3.8-27B (48 Gated DeltaNet layers, 16 attention layers) on one NVIDIA DGX Spark. It drafts a token tree from a block drafter and a lookup drafter, verifies it in one pass, and commits the accepted path's state in closed form. StairCut, a router, cuts the tree each round to the size with the highest expected committed tokens per millisecond on a measured, stepwise table of the verify cost. The model is quantised to 4-bit NVFP4 weights and published. On single-request decoding (256 prompt tokens, 256 new tokens, 50 requests, greedy), StairCut reaches 49.89 tokens per second against 47.13 and 44.80 for the same drafter family at fixed blocks of 8 and 16, and 13.69 for plain greedy decoding; on a teacher-forced bench of 25 workloads with identical committed text it is above the fixed block 16 on 23 and above block 8 on 22. Agreement with plain greedy decoding is reported as measured: of 30 outputs, 17 are identical up to the end of the answer and 13 diverge at a near-tie of the two best tokens (at most one unit in the last place), where the batched verify rounds differently. Cached recurrent-state anchors cut the time to first token of a repeated 128k-token prompt from 208.3 s to 1.1 s.