Reinforcement Learning based AP Selection in Cell-Free Massive MIMO Systems with Phase-Noise

In this paper, we propose an Advantage Pointer Critic (APC)-based access points (APs) selection algorithm in Cell-Free massive MIMO (CF-mMIMO) systems. In such system, a Central Processing Unit (CPU) coordinates multiple distributed APs to jointly serve user equipments (UEs), and the phase noise is applied during the signal propagation. To enhance the downlink sum rate, we derive the deterministic equivalence expression of the general ergodic achievable rate by rigorous mathematical analysis. Based on the achievable rate, we construct the corresponding optimization problem based on AP selection and refomulate it as a Markov decision process (MDP). Furthemore, we introduce the actor-critic network with pointer network as its backbone and propose an APC algorithm to address the MDP. Comprehensive simulation results demonstrate that the proposed APC-based AP selection algorithm outperforms the benchmark algorithms, highlighting its effectiveness in enhancing the downlink general ergodic achievable rate within practical CF-mMIMO system.