Aerial Hybrid STAR-RIS-Aided Integrated Sensing and Communication System via Deep Reinforcement Learning

This paper investigates an aerial hybrid simultaneously transmitting and reflecting reconfigurable intelligent surface (STAR-RIS)-aided integrated sensing and communication (ISAC) system, where the hybrid STAR-RIS is mounted on an unmanned aerial vehicle (UAV) to assist in communication with ground terminals and concurrently probing for echo signals from targets of interest. To maximize the sum-rate of communication users while ensuring the sensing quality of sensing targets, a joint optimization problem involving the UAV trajectory, STAR-RIS transmission and reflection coefficients, and base station beamforming is formulated. However, there is an unpredictable nature in the environment, such as changes in locations of the users, targets, and UAV, making it difficult to be solved by traditional optimization methods. To address this difficulty, the formulated problem is first modeled as a Markov decision process, and then a deep reinforcement learning framework based on proximity policy optimization (PPO) algorithm is proposed to solve this intractable problem. Finally, simulation results demonstrate that the proposed PPO-based algorithm is superior to the benchmark algorithms in terms of the convergence performance. Moreover, compared to other schemes, the average sum-rate performance is improved by the proposed scheme.