Automated Resource Orchestration for Large-Scale Heterogeneous Applications

Designing a scheduling algorithm that eliminates dependence on manual rule design while exhibiting excellent scheduling performance and adaptive capabilities has become a key research direction for heterogeneous resource scheduling for large-scale heterogeneous applications. To address these challenges, we innovatively introduce large language model (LLM) technology and propose a cluster scheduling algorithm based on LLMs. First, we model the heterogeneous resource scheduling scenario for AI tasks as a Markov decision process, defining the state transitions, action space, and reward mechanism of the cluster scheduling process. Then, we design an LLM-based scheduler, which integrates a pre-trained large language model and is trained using an offline reinforcement learning paradigm: historical cluster scheduling data is collected by employing a baseline algorithm for cluster scheduling, and this data is then used to fine-tune the model. To validate the effectiveness of this scheduler, we construct a simulated cluster environment using real-world cluster scheduling records from production environments and compare the scheduling performance of various algorithms, including the LLM-based scheduler, within this simulated environment. The experimental results demonstrate that the proposed LLM-based scheduler not only outperforms the baseline algorithms (reducing the average job completion time by 3.8%–17.0% on the test set compared to the best-performing baseline) but also exhibits stronger adaptability to different scheduling scenarios.