Deep Reinforcement Learning for Movable Antenna-Assisted Cell-Free Networks

In this paper, we propose a movable antenna (MA)-assisted cell-free communication network to serve multiple users. The proposed MA is capable of moving in a three-dimensional region, and making full use of the spatial degrees of freedom (DoFs) to adjust the channel condition. As such, higher spatial diversity gains are achieved at the transmitters, thereby significantly enhancing the signals received at the users. With multiple MAs per access point (AP), we maximize the sum-rate of all users by jointly optimizing the MA positions and digital precoders. Then, we propose a novel deep reinforcement learning (DRL)-based algorithm, named multi-agent twin delayed deep deterministic policy gradient (MATD3)-team minimum mean square error (TMMSE) to solve this non-convex problem efficiently. Simulation results demonstrate that, through the proposed MATD3-TMMSE algorithm, the joint design of cell-free networks and MA architecture can effectively improve the sum-rate and reduce the network cost compared to existing fixed-position antenna (FPA) networks and traditional precoding algorithms.