Multi-Agent DRL for RIS Partitioning, Beam Selection, and Power Control in MIMO-NOMA System

Reconfigurable intelligent surface (RIS) partitioning offers a strategic solution for serving multiple users equipment (UEs) simultaneously in blockage-prone wireless environments, leveraging multiple-input multiple-output (MIMO) and non-orthogonal multiple access (NOMA) technologies within the millimeter-wave (mmWave) spectrum. However, deploying RIS partitioning (RISP) in MIMONOMA is hindered by the exacerbated computational complexity involved in acquiring accurate channel state information (CSI). This paper proposes a novel learning framework based on multi-agent deep reinforcement learning (MA-DRL) that maximizes the sum rate of the RISP-aided MIMO-NOMA system without requiring UEs’ CSI. The proposed framework jointly optimizes RIS phase shifts, number of RIS partitions, UEs beamformers, and NOMA power allocation (PA), transforming the challenge into a high-dimensional combinatorial optimization problem. The MA-DRL algorithm integrates double deep Q networks (DDQN) and deep deterministic policy gradient (DDPG) agents, where each UE acts as a DDQN agent optimizing its beamformer, while the RIS serves as a DDPG agent handling partitioning and power control. An experimental testbed is developed to gather real-world data for training and evaluation. Results show that the MA-DRL algorithm closely approaches optimal performance, trailing the exhaustive search by only 8%, while reducing complexity by 95% and improving the sum rate by an average of 18% compared to traditional full RIS setups.