Joint Optimization of Beam Selection and Power Control in Massive MIMO Using a Surrogate Model

Broadcast beam design and power control are essential for enhancing the network coverage, improving the quality of service (QoS) and reducing the energy consumption in massive multiple-input-multiple-output (MIMO) communications. To improve the broadcasting performance and decrease the power consumption in dynamic scenarios with varying user numbers and distributions, we leverage deep reinforcement learning (DRL) to jointly optimize the beam selection and power control policy, and propose a multi-agent DRL (MA-DRL) framework to address the extremely high action dimension brought by the non-convex combinational multi-objective optimization problem. To reduce the cost of performance fluctuations during the exploration of DRL, we construct a data-driven surrogate model (SM) as a virtual environment for the initial training phase, while using an empirical baseline scheme to ensure acceptable real-time performance. Simulation results demonstrate that the SM-enabled MA-DRL approach not only enhances the coverage and reduces the power consumption, but also enables safe exploration and rapid adaptation to varying user numbers and distributions. Moreover, since the optimization algorithm can interact with the SM much more quickly than the real network, a faster convergence speed can be achieved with the help of the SM.