Enhancing Interference Mitigation via Hybrid Action Space Control Deep Reinforcement Learning

Multi-agent deep reinforcement learning (DRL) has been utilized for interference mitigation in multi-cell networks, employing either discrete or continuous action spaces. A DRL algorithm with hybrid action spaces can combine discrete and continuous action spaces, which is natural when actions include both discrete and continuous components. By employing continuous power allocation but discrete beamforming selection, this paper introduces a hybrid multi-agent DRL algorithm for enhancing interference mitigation in millimeter-wave multi-cell networks. In this multi-agent DRL, each base station is equipped with an agent that receives the signal-to-interference-plus-noise ratio (SINR) from the associated users. A central controller collects these SINRs to generate and assign uniform rewards/penalties to all agents so that they can update their models locally. Hence, the algorithm adopts centralized reward generation but distributed model updating. This reduces the communication between the agents and the central node to SINR and reward exchange. Simulation results confirm that this hybrid action space interference management algorithm outperforms approaches utilizing either discrete or continuous action spaces in terms of the system sum-rate. Additionally, the proposed framework exhibits scalability with the number of cells.