DFRL-DS: A Diffusion-based Reinforcement Learning Algorithm in Discrete Actions for Base Station Energy-saving Control

The exponential growth of mobile traffic is escalating energy usage in global mobile networks. Newer cellular technologies like 5G, while offering increased throughput and reduced latency, also require more power for denser base station layouts and more demanding processors. Improving energy efficiency is critical for the design of future mobile networks. In response to these challenges, we propose a novel algorithm combining reinforcement learning (RL) and diffusion models in discrete actions (DFRL-DS) for energy-saving control of base stations. Our proposed algorithm, DFRL-DS, stands at the forefront of this endeavor, offering a sophisticated control strategy that is both adaptive and efficient. We represent the forward process (noise addition) of the diffusion model as the training process of the policy network, and the reverse process (denoising) of the diffusion model as the decision- making process of the policy network. In addition, we devise a new mechanism for the reward function of RL. Through experiments, our algorithm achieves significant energy conservations while maintaining an acceptable level of Quality of Service (QoS). On average, DFRL-DS saves 49% to 59% energy compared to baseline algorithms and 20% to 42.5% compared to other popular deep reinforcement learning algorithms.