Multi-Agent Deep Reinforcement Learning for Integrated Sensing and Communication in RIS-aided UAV Networks

Integrated Sensing and Communication (ISAC) is regarded as a promising approach to improve the original performance for unmanned aerial vehicle (UAV) networks. Nevertheless, it is still limited by the UAV's energy constraints and dynamic links with user equipment (UE). To address these issues, we attempt to deploy the Reconfigurable Intelligent Surface (RIS) for signal reflection, improving sensing and communication performance. Additionally, employing downlink-uplink decoupling (DUDe) can enable each UE to associate with diverse UAVs for downlink (DL) and uplink (UL), further enhancing transmission quality. Therefore, we study the RIS deployment, decoupled UE- UAV association and trajectory design for a RIS-aided UAV network. A joint optimization problem is formulated for maximizing the sum rate in UL and DL. Specifically, we transform the joint problem as a Markov Decision Process, and employ a distributed multi-agent deep reinforcement learning (MADRL) approach to select policies. Moreover, we develop a robust Proximal Policy Optimization (PPO) algorithm to train the AC networks, wherein the Random Environment Distribution is utilized for adapting to varying scenarios and we design an intrinsic reward to expand UAV s' exploration range. Simulation results validate the feasibility and superiority of the RED-PPO approach through comparative analysis.