Throughput Maximization in NOMA Enhanced RIS-Assisted Multi-UAV Networks: A Deep Reinforcement Learning Approach

In this paper, a novel reconfigurable intelligent surface (RIS) aided communication system with multiple unmanned aerial vehicles (UAVs) is investigated. To achieve better system performance, we employ the non-orthogonal multiple access (NOMA) technique, and consider imperfect successive interference cancellation (SIC) at each user equipment (UE). To maximize the system throughput, we jointly optimize the three-dimensional (3D) trajectories of UAVs, the power allocation strategy, and the phase shift of the RIS. Due to the movement of UEs and UAVs, the formulated problem is difficult to solve. To address the formulated problem, we propose a two-step approach, named throughput maximization by trajectories design, power allocation and phase shift optimization (TM-TDPAPO). Specifically, a <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">K</i>-means based UE clustering algorithm is adopted to group UEs. Then, a double deep Q-network (DDQN) approach is utilized to deal with the formulated problem. Extensive simulations are conducted, and results demonstrate that: (1) the TM-TDPAPO algorithm is effective in improving the system throughput, especially, the proposed approach achieves approximately 57<inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\%$</tex-math></inline-formula> higher throughput gain compared to the conventional DQN algorithm; (2) by applying the imperfect SIC at each UE, the proposed scheme yields 23<inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\%$</tex-math></inline-formula> performance gain than orthogonal multiple access (OMA) case; (3) the considered system with the deployment of RIS benefits from 29<inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\%$</tex-math></inline-formula> throughput gain over the system without RIS.