A Transformer-embedded Reinforcement Learning for Computing Power Scheduling in Data Centers

The growing demand for computing power driven by advancements of Cloud Computing has posed significant challenges to data centers. As a solution, distributed data centers have emerged, so the optimization of edge computing and resource allocation is of great significance for research. However, the increasing complexity of scheduling rules and the need for flexible load management, along with the growing demands for real-time control and robustness, present considerable challenges to traditional modeling and optimization methods. To address this problem, this study models the task-level computing power scheduling optimization problem and formulates it as a Markov Decision Process (MDP). Then a Transformer-embedded Proximal Policy Optimization (PPO) framework is developed as the solution methodology. The Transformer component captures temporal relationships among tasks and processes variable-length states, which is integrated into PPO to optimize the policy. Numerical experiments on the Alibaba Cluster Trace V2017 dataset show that the proposed method is compatible with various lower-level control policies, effectively reducing energy consumption while maintaining latency within predefined thresholds, thus achieving a controllable trade-off between energy cost and job latency.