Multi-Cell User Association and Resource Allocation in MU-MIMO Systems via Multi-Agent Reinforcement Learning Framework

In this paper, we introduce a novel user association (UA) and resource block group (RBG) allocation (RA) method utilizing multi-agent reinforcement learning (MARL) for a multi-user multiple-input multiple-output (MU-MIMO) downlink system. Unlike traditional MARL radio resource management (RRM) approaches, which utilize user equipment (UE) as learning agents, base stations (BS) are deployed as agents for practical consideration. However, this will significantly enlarge the action space and bring about action constraint violation problems. We employ dual-actor neural networks to separate UA and RBG allocation actions, thereby effectively reducing the joint action space and accelerating exploration. In addition, a Q-value-rank-based action projection algorithm is proposed to address the cross-agent coupling constraints. The simulation results demonstrate that the proposed MARL framework with action projection outperforms other baselines in terms of RRM performance.