Deep Reinforcement Learning-Based Interference Mitigation in mmWave Multi-Cell Networks With Implicit CSI

In this paper a deep reinforcement learning (DRL) based interference mitigation is proposed for multi-cell downlink in millimeter-wave (mmWave) networks. Unlike existing approaches that require exchanging full channel state information (CSI) among cells, the proposed framework exploit implicit CSI by sharing only the received signal-to-interference-plus-noise ratio. The proposed DRL framework is leveraged through two different algorithms which manage interference through joint beamforming selection and power control in multi-cell mmWave networks. First, a deep Q-network algorithm is developed to select discrete power and beamforming vectors, leveraging implicit CSI without requiring full CSI sharing. Then, to address the limited flexibility of discrete actions in dynamic environments, an actor-critic-based DRL approach is introduced which enables continuous and fine-grained control of transmission parameters. The proposed algorithms demonstrate strong scalability with respect to the number of cells. Simulation results in mmWave networks with two to seven cells confirm that the algorithms effectively learn near-optimal transmission strategies to maximize spectral efficiency in line-of-sight, non-line-of-sight, and mixed environments even without full CSI sharing.