Deep reinforcement learning based communication resource allocation driven by radar point cloud for urban air mobility

In the future smart cities, unmanned aerial vehicles (UAVs) or electric vertical take-off and landing aircraft (eVTOL) are widely employed for urban air mobility (UAM). Considering such real-world scenarios, a deep reinforcement learning based communication resource allocation method is proposed for UAVs to provide communication services for eVTOL swarms to ensure their reliable communication and safe operation. To save energy consumption, UAVs can ride on a moving interaction station (MIS), such as an urban bus. By using UAV trajectory control and communication power allocation, a joint fair optimization problem is formulated to maximize the channel capacity while optimizing radar sensing performance. To address the optimization problem, a Point Cloud based deep Q-network (PCDQN) algorithm is proposed. It contains a point neural network that can determine the action space of the UAV directly originating from the three-dimensional (3D) radar point clouds, and a deep reinforcement learning based decision model for deciding the action from action spaces. Simulation results demonstrate that the proposed method exhibits competitive performance compared to the benchmarks.