A learning-based algorithm for turn-based orbital pursuit-evasion problem with reaction-time delay

In this paper, we propose an artificial intelligence-based methodology to investigate the impulsive orbital pursuit-evasion problem, taking into account the often-neglected factor of reaction time delay. This study defines this scenario as a Delayed Turn-based Orbital Pursuit-Evasion Game (DT-OPEG) and establishes relevant concepts and definitions based on orbital dynamics and game theory . Subsequently, considering several constraints inherent in real-world missions, including nonlinear orbital dynamics, maneuvering capabilities, fuel reserves, and mission duration, we formulate the problem modeling for DT-OPEG. To address the complexity of this problem, especially the challenge of incorporating action delays, we propose a Delayed Turn-based Multi-Agent Deep Deterministic Policy Gradient (DT-MADDPG) algorithm. The establishment process of this algorithm includes establishing a Turn-based Markov Decision Process (T-MDP) model with reaction-time delay, constructing a turn-based training framework, developing a network architecture based on MADDPG, and designing reward functions. Finally, simulation analyses are conducted for both two-dimensional and three-dimensional DT-OPEG scenarios, confirming the effectiveness of the proposed algorithm and demonstrating the winning mechanism in this type of game. • A Delayed Turn-based Orbital Pursuit-Evasion Game (DT-OPEG) model is constructed. • An efficient learning-based algorithm (i.e. DT-MADDPG) for DT-OPEG problem is designed. • The Turn-Based Markov Decision Process model is proposed for DT-OPEG. • A training framework considering reaction time delay for improving MADDPG is established. • The impact of reaction-time on the winning probabilities is analyzed.