A Spatiotemporal Graph Reasoning Approach for Pursuit-Evasion Game With Communication Limits

Unmanned aerial vehicle (UAV) swarms performing encirclement tasks require reliable communications to coordinate strategies to capture the fleeter target. However, dense urban buildings can obstruct signals, causing communication disruptions, observation fluctuations, and decreased strategy efficiency. To address this problem, we propose the Relational Graph Attention Twin Delayed DDPG (RGATD3) approach. In particular, we adopt a spatiotemporal graph to represent complex state information, and a graph prediction network to anticipate missing communication data, effectively stabilizing observation fluctuations. In order to improve decision-making in complex environments, a relational graph learning mechanism is introduced, where Graph Attention Networks (GATs) extract agent features based on specific relational types, and a multi-head self-attention mechanism aggregates these features. Combined with localized detection, this approach enhances decision-making from complex observation. Additionally, well-tailored reward functions further encourage agents to restore communication and cooperation. The results indicate that RGATD3 outperforms existing state-of-the-art methods, increasing encirclement success and reducing capture time under fluctuating communication conditions.