Distributed Noncoherent Joint Transmission Based on Multi-Agent Reinforcement Learning for Dense Small Cell MISO Systems

This paper investigates the sum rate maximization problem in dense small cell (DSC) networks, where multi-antenna small cell base stations (SBSs) serve single-antenna users over a shared frequency band using noncoherent joint transmission (JT). While noncoherent JT enhances capacity by enabling users to receive signals from multiple coordinated SBSs, its practical deployment is hindered by the nonconvex and NP-hard nature of the optimization problem, along with the high-dimensional continuous optimization variables. Existing optimization-based approaches, despite achieving near-optimal performance, require global channel state information (CSI) and multiple iterations, making them impractical for DSC networks. To overcome these challenges, we first prove that the optimal beamforming structure is the same for both the power minimization problem and the sum rate maximization problem, and then mathematically derive the optimal beamforming structure for both problems by solving the power minimization problem. The optimal beamforming structure can effectively reduces the variable dimensions. By exploiting the optimal beamforming structure, we propose a deep deterministic policy gradient-based distributed noncoherent JT scheme to maximize the system sum rate. In the proposed scheme, each SBS utilizes global information for training and uses local CSI to determine beamforming vectors. Simulation results demonstrate that the proposed scheme achieves comparable performance with considerably lower computational complexity and information overhead compared to centralized iterative optimization-based techniques, making it more attractive for practical deployment.