Joint Trajectory Design and Phase Shift Optimization for Multi-RIS-Assisted UAV Relay Network Using Deep Reinforcement Learning

This article investigates an uncrewed aerial vehicle (UAV) relay network in the urban Internet of Things (IoT) environment designed to facilitate communications for ground terminals (GTs) by transmitting their data to a remote base station (BS) using the decode and forward (DF) protocol, where the orthogonal frequency division multiple access (OFDMA) is adopted for the GTs’ transmissions. To address the challenges posed by building obstructions on radio propagation, we incorporate the multiple reconfigurable intelligent surfaces (RISs) to improve the quality of the GTs-UAV and UAV-BS links. The primary objective is to maximize the sum rate of all the GTs through the joint optimization of UAV trajectory, GTs transmit power, subchannel allocation, and multi-RIS phase shifts. To this end, we reformulate the optimization problem as a Markov decision process (MDP) and propose a deep reinforcement learning (DRL) approach for addressing our formulated problem, called the soft actor critic-based subchannel allocation and phase shift optimization with trajectory design (SAC-CAPSTD) algorithm. By continuously interacting with the environment, the proposed system refines its policy to determine the optimal UAV flight trajectory and the subchannel allocation strategy for GTs within their transmit power constraints. Concurrently, a discrete phase shift optimization method is implemented to adjust the phase shift for each RIS element. Finally, numerical results confirm that the proposed SAC-CATDPS algorithm can significantly achieve higher sum rate of all the served GTs and exhibit better convergence performance compared with the benchmark DRL-based algorithms.