Collaboration Strategies for Two Heterogeneous Pursuers in A Pursuit-Evasion Game Using Deep Reinforcement Learning

We investigate a pursuit-evasion game taking place in an unbounded three-dimensional space, where a flexible pursuer with hybrid dynamics collaborates with a fast pursuer and aims to capture a flexible evader within a finite time. The key feature of this problem lies in the hybrid dynamics of the flexible pursuer, which can change its dynamics once during the game and switch to a fast pursuer with increased speed but lower maneuverability. To address this challenge, we devise a hybrid strategy based on the soft actor-critic framework, tailored specifically for the flexible pursuer, which encompasses both maneuvering and switch tactics. We introduce a switch factor to the input of the actor network and incorporate switch actions to further expand the action space. These additions enable the flexible pursuer to execute maneuvering actions and determine a moment to switch to a fast pursuer. The reward function is designed to account for related angle, altitude, speed, and sparse reward. Through extensive ablation experiments conducted in a simulated environment, we demonstrate the efficacy of our algorithm in facilitating the learning of hybrid strategies for the flexible pursuer, resulting in significantly improved capture rates compared to alternative methods.