Distributed DQN-Based Frequency Block-Dependent Sleep Control of Base Stations for Improving Energy Efficiency Both in Up and Downlinks

This paper investigates a frequency block-dependent sleep control method of base stations (BSs) based on a deep Q-network (DQN) that tracks changes in user distribution aiming at the improvement in energy efficiency both in up and downlinks. Different from the conventional centralized control, which represents the entire system with a single action value function, the proposed method assumes that each BS is an agent and each BS has its own action value function, making the DQN-based BS activation control applicable to large-scale networks. To mitigate the negative impact of the exponential increase in the total number of actions due to the individual sleep control of all the frequency blocks for up and downlinks, we propose a divided model between uplink and downlink. Furthermore, a restricted action model is employed in the proposed method to accommodate the deep sleep state. Computer simulation shows the effectiveness of the proposed method by comparing the achievable performance of the proposed method using various DQN models with those of conventional probabilistic control method.