Multi-Agent RL for Sleep Mode and Antenna Configuration With User Offloading Under Dynamic Traffic in Massive MIMO Networks

In this paper, we focus on minimizing the total energy consumption of multi-cell massive multiple-input multiple-output (MIMO) networks while simultaneously guaranteeing user quality of service (QoS). This is achieved by optimizing the multi-level advanced sleep modes (ASM), antenna switching, and user association of the base stations (BSs). Due to the interdependence of user association and inter-cell interference in the network, collaborative efforts among individual BSs become imperative. The problem is modeled as a decentralized partially observable Markov decision process (DEC-POMDP) and a multi-agent proximal policy optimization (MAPPO) algorithm is proposed to obtain a collaborative BS control policy. Simulation results demonstrate that the obtained policy can significantly improve network energy efficiency, adaptively switch the BSs into different depths of sleep, reduce inter-cell interference, and maintain good QoS compared to the two benchmark algorithms. The results also validate that enabling user offloading among BSs can improve both user QoS and system performance. The superiority of MAPPO is further affirmed by comparing it with the single-agent deep Q network (DQN) algorithm.