The Price of Order in Disaggregated Inference
"The Price of Anarchy in Disaggregated Inference" (Georgiou, 2026) modeled disaggregated LLM inference as three coupled games (allocation, cache placement, and routing) and posed the allocation game analytically, because no fleet under study could vary the prefill/decode split at runtime. We build such a fleet and measure what that coordination costs. We present Narwhal, an open-source orchestration framework in which a worker's role is a label the controller rewrites while the weights stay resident, so a re-split costs seconds where a drain-and-reprovision would cost minutes. A five-arm topology walk drives the target split across a six-node AMD MI355X fleet serving Kimi-K3. Across two walks (40,661 offered requests per arm), two hot-swap controllers attain 95.3% and 98.5% client-side against a 15 s time-to-first-token and 60 ms time-per-output-token budget, against 93.4% for a pinned static split and 32.5% for a non-disaggregated aggregate. Charging the same planner 300 s of drain per re-role drops it to 54.7%, and all 12,760 of its refusals fall inside actuation windows. We formalize the price of order (the actuation time, hunting error, and denied admissions a coordinating mechanism pays to hold a moving equilibrium) and show it is small when actuation is a label write and exceeds the anarchy it removes when actuation costs minutes. Companion campaigns price each game directly. The software and the complete measurement record are archived and cited.
Authors
- Athos Georgiou
Publication Details
- Journal
- arXiv (Cornell University)
- Published
- 2026-08-24
- DOI
- https://doi.org/10.5281/zenodo.22084806
- Primary Topic
- Parallel Computing and Optimization Techniques
- Type
- preprint