Reinforcement-Learning-Based AAV 3-D Target Tracking and Digital-Twin-Assisted Collision Avoidance With Integrated Sensing and Communication

The flexibility and maneuverability of unmanned aerial vehicles (UAVs) lend themselves to tracking users and operating as an aerial base station carrying out communication enhancement functionality. A core challenge neglected by most existing works is that the true-but-unknown obstacles can jeopardize UAV flight security and shadow its communication links with users, resulting in poor achievable rate and high collision risks. In this paper, a deep-reinforcement-learning (DRL)-based UAV target tracking and digital-twin (DT)-assisted collision avoidance method is proposed to optimize UAV’s communication performance while tracking moving users. Toward this end, Twin Delayed Deep Deterministic policy gradient (TD3) as a novel and policy-based DRL algorithm is used to construct an agent responsible for adaptive deciding UAV flying control actions. To efficiently detect unknown obstacles in a flight environment, an orthogonal frequency division multiple (OFDM)-based integrated sensing and communication (ISAC) system is investigated, endowing UAV’s agent with real-time obstacle distance. Finally, we present a DT obstacle model construction mechanism and integrate it with TD3 agent training. The extensive simulations demonstrate the reward convergence of the TD3 algorithm and the communication improvement with stable user tracking and reliable collision avoidance, compared with conventional approaches.