Model-Based Reinforcement Learning for Energy Efficiency Optimization in Small-Cell Networks

With the increasing density of base stations, the energy consumption of the 5-th generation mobile networks (5G) has become a serious issue and has attracted attention. To address this challenge, a novel model-based reinforcement learning (MBRL) algorithm for joint base station advanced sleep mode (ASM) and power control is proposed, aiming at optimizing the energy efficiency of small-cell networks. Taking the data rate and delay requirements of users into consideration, we develop a small-cell network model with diverse types of users. The energy efficiency optimization problem is formulated as a Markov decision process (MDP) and a model-based reinforcement learning approach is designed to solve this problem. Simulation shows that the proposed algorithm converges quickly and achieves higher energy efficiency (EE) and lower packet loss rates, compared with baseline methods, especially in high-load and high-interference conditions. This study provides both theoretical and practical insights for the green development of 5G and future communication networks.