Flexibility Resource Services and Electricity Cost Optimization Oriented Control Strategy of Data Centers Based on Hierarchical Reinforcement Learning

As the core of digital infrastructure, the exceptionally rapid development of data centers (DCs) faces serious challenges due to their high electricity costs. Traditional approaches treat computational task scheduling separately from different physical control mechanisms, such as server group management, overlooking the synergistic potential between the two aspects. To address this problem, this paper proposes a computational–physical collaborative optimization model that realizes spatiotemporal task migration on the computational side and adaptive parameter regulation of IT equipment and cooling devices on the physical side. In response to the lack of global coordination in conventional distributed optimization, a two-layer partially observable Markov game (POMG) is constructed to unify global cooperative decision-making and local autonomous control. On this basis, the hierarchical multi-agent deep deterministic policy gradient (H-MADDPG) algorithm is designed by introducing task priority ranking and a variable-dimension action mask mechanism, which effectively handles the discrete–continuous hybrid action space and adapts to the dynamic variation in action dimensions caused by uncertain task arrivals. Comparative experiments with various benchmark schemes are conducted to verify the effectiveness and superiority of the proposed strategy in total cost, power usage effectiveness (PUE), resource utilization, and load balancing.