Integrated Sensing and Communications for UAV Assisted Internet of Things Based on Deep Reinforcement Learning

As an emerging technology, Integrated Sensing and Communications (ISAC) is expected to be applied to uncrewed aerial vehicle (UAV)-assisted Internet of Things (IoT) systems to sense and acquire information from IoT devices in complex environments. In this paper, we design a UAV-ISAC assisted IoT system, where the UAV senses the IoT nodes in unknown locations and simultaneously forwards the sensing information to the ground base station (BS). The UAV is required to complete the sensing of all IoT nodes before returning to the starting position. However, several challenges arise in designing this system, including unknown IoT node positions, a non-convex optimization problem, constrained resources, a dynamic environment, and complex resource allocation. We propose using radar estimation rate as the measure for evaluating the UAV sensing performance. We formulate a UAV trajectory optimization problem that aims to maximize the radar estimation rate and minimize the UAV flight time, while adhering to constraints on radar sensing performance, information rate, and sensing all the nodes. However, solving this problem using standard optimization techniques is challenging due to the lack of information about the locations of IoT nodes and the non-convex nature of the problem. To address this challenge, we propose an algorithm by adopting multi-step learning Dueling Double Deep Q-learning network (DDQN) based on deep reinforcement learning (DRL) to solve this intractable optimization problem. The simulation results demonstrate that our proposed algorithm achieves stable convergence under different constraint thresholds. Furthermore, we verify the superiority of our algorithm by comparing it with other benchmark algorithms.