Theoretical Foundation of Hard Top-k Primal Routing
A pervasive misconception in deep learning router design is that discrete Top-k selection operates as an ad-hoc, non-differentiable heuristic necessitating continuous relaxations such as softmax smoothing. Here, we formalize the token-to-expert dispatch mechanism through convex optimization and network flow theory, demonstrating that the hard Top-k operator over duality-adjusted affinities is the exact, closed-form solution to a linear program, incurring a zero integrality gap.
Authors
- Carl Johan Rehn (ORCID: https://orcid.org/0009-0002-7844-0770)
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-11-16
- DOI
- https://doi.org/10.5281/zenodo.24284083
- Primary Topic
- Advanced Neural Network Applications
- Type
- article
- Field-Weighted Citation Impact
- 0.00