Spatiotemporal-Aware Deep Reinforcement Learning for Multi-UAV Cooperative Coverage in Emergency Deterministic Communications

In post-disaster reconstruction and rescue work, unmanned aerial vehicles (UAVs) are often used as aerial base stations to provide communication services for ground users (GUs), owing to their flexibility and controllability. The premise for the efficient completion of cooperative rescue tasks is the deterministic transmission of data, that is, to ensure that the end-to-end delay of data (Time Sensitive Flow, TSF) is bounded and the jitter is stable. However, a single UAV is insufficient to fully cover the entire target area. To obtain an efficient multi-UAV cooperative control coverage strategy that directs multiple UAVs to maximize the number of coverage tasks in a cooperative manner, the centralized controller must jointly consider the spatial positions of both the mission and the UAVs, as well as the effective transmission time windows for TSFs. Towards this end, we propose a spatiotemporal aware deep reinforcement learning (STA-DRL) method. The proposed method 1) The transformer is used to extract spatiotemporal features from environmental states to improve the generalization representation ability of the model; 2) Prioritized Experience Replay Buffer helps accelerate model convergence; 3) A coverage strategy is formulated under the guidance of two Transformer models. Simulation results show that the proposed method is superior to existing methods in terms of task completion coverage quantity.