BandPilot: Towards Performance-Aware GPU Dispatching in AI Clusters

In modern AI clusters, the performance of parallel computing critically depends on the communication bandwidth among the allocated GPUs. Existing GPU dispatchers rely on static, locality-based heuristics that prioritize resource compactness. However, this approach proves suboptimal as it overlooks key performance factors. We introduce BandPilot, a performance-aware GPU dispatching system that maximizes effective communication bandwidth while eliminating manual topology configuration. BandPilot addresses the dispatching problem in two stages: i) a hierarchical Transformer model that accurately predicts end-to-end NCCL bandwidth for arbitrary GPU configurations, and ii) a lightweight search algorithm combining an Equilibrium-driven Heuristic with Pruned Tree Search to identify near-optimal GPU allocations. We implement and evaluate BandPilot on a real-world AI cluster and in trace-driven simulations. Experimental results demonstrate that BandPilot consistently achieves high bandwidth efficiency with negligible overhead, outperforming state-of-the-art topology-aware dispatchers by $12\sim 31\%$ across diverse AI cluster configurations.