Deep Reinforcement Learning-based Evasive Maneuvering in Within-visual-range Aerial Combat

In this study, a one-on-one within-visual-range (WVR) air-to-air combat scenario is considered, where a pursuer is controlled using a virtual-pursuit-point (VPP)-based guidance law, and an evader is trained using a deep-reinforcement-learning approach based on the TD3 algorithm. The pursuer is implemented using the conventional VPP framework commonly employed in previous studies, adopting both lead and lag pursuit geometries. In addition, the pursuer dynamically generates tactical pursuit points in real time according to the engagement geometry and the opponent’s kinematic state. Conversely, the evader is designed using the Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithm to learn a continuous-control policy network suitable for maneuvering in close-range aerial engagements. To ensure tactically meaningful and survival-oriented behavior, the reward function and termination conditions are formulated by incorporating essential WVR constraints, including gun engagement range and flight envelope limitations. As a result, the evader learns evasive maneuvers that go beyond simple heading changes and executes coherent, tactically valid actions aimed at maximizing survivability.