TokenRouter Is the Scheduler You Need: A Self-Learning Multipath Scheduler for Quota-Constrained LLM Serving

Large language model (LLM) gateways increasingly route requests over multiple heterogeneous upstream channels. In this setting, each channel may have an independent quota state, latency distribution, transient availability window, and opaque server-side throttling policy. Conventional static round-robin and single-lane fill-first strategies are simple to deploy, but they can repeatedly select degraded channels and amplify local upstream failures into burst failures. This paper presents TokenRouter, a self-learning multipath scheduler for black-box, quota-constrained LLM serving. TokenRouter maintains a compact active channel set, learns channel health from online feedback, performs capacity-aware admission before dispatch, evicts hard-failing channels immediately, and applies group-level circuit breaking when correlated failures appear. The active set starts from a minimum healthy subset and expands or shrinks according to recent requests per minute, rather than using a fixed global fan-out. On a production trace with 12,446 dispatch events, static scheduling shows severe failure clustering: the worst five-minute window has only 1.91% success rate, while several stable windows reach 100%. A short online validation trace after deploying the multipath health filter improves the worst five-minute success rate to 76.06%. A next-day online trace with 20,283 requests reaches 92.34% overall success rate; after excluding the cold-start exploration period, the stable segment contains 18,080 requests with 97.80% success rate and an 88.24% worst-window success rate. The results indicate that quota-aware active-set scheduling can substantially reduce burst-failure risk under opaque upstream constraints.