The Price of Order in Disaggregated Inference

"The Price of Anarchy in Disaggregated Inference" (Georgiou, 2026) modeled disaggregated LLM inference as three coupled games (allocation, cache placement, and routing) and posed the allocation game analytically, because no fleet under study could vary the prefill/decode split at runtime. We build such a fleet and measure what that coordination costs. We present Narwhal, an open-source orchestration framework in which a worker's role is a label the controller rewrites while the weights stay resident, so a re-split costs seconds where a drain-and-reprovision would cost minutes. A five-arm topology walk drives the target split across a six-node AMD MI355X fleet serving Kimi-K3. Across two walks (40,661 offered requests per arm), two hot-swap controllers attain 95.3% and 98.5% client-side against a 15 s time-to-first-token and 60 ms time-per-output-token budget, against 93.4% for a pinned static split and 32.5% for a non-disaggregated aggregate. Charging the same planner 300 s of drain per re-role drops it to 54.7%, and all 12,760 of its refusals fall inside actuation windows. We formalize the price of order (the actuation time, hunting error, and denied admissions a coordinating mechanism pays to hold a moving equilibrium) and show it is small when actuation is a label write and exceeds the anarchy it removes when actuation costs minutes. Companion campaigns price each game directly. The software and the complete measurement record are archived and cited.

Authors

Publication Details

Journal
arXiv (Cornell University)
Published
2026-08-24
DOI
https://doi.org/10.5281/zenodo.22084806
Primary Topic
Parallel Computing and Optimization Techniques
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
preprint

The Price of Order in Disaggregated Inference

Athos Georgiou
arXiv (Cornell University)
Parallel Computing and Optimization Techniques
preprint

The Price of Order in Disaggregated Inference

Athos Georgiou
preprint en

Abstract

No abstract available for this paper.

arXiv (Cornell University)
Parallel Computing and Optimization Techniques
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.