Research on air combat maneuver decision-making method based on expert-augmented reinforcement learning

Artificial intelligence technology has an increasingly far-reaching impact in modern warfare, and improving the intelligence level of unmanned combat aircraft has become a key issue in future warfare. UAVs driven by deep reinforcement learning algorithms are able to make decisions autonomously by perceiving the environment in the combat environment. In this paper, we adopt deep reinforcement learning policy optimization method to study the UAV maneuver decision-making problem in continuous action space. First, we utilize the Harfang engine to establish a reinforcement learning simulation model for UAVs in continuous action space. To address the problems of long training time and low data utilization faced by traditional reinforcement learning methods, this paper proposes an expert knowledge-guided UAV maneuvering decision-making algorithm-Expert-Augmented TD3 algorithm, which combines the experts' experience for assisted exploration in the exploration phase, and utilizes the autonomous exploration for updating the learning strategy in the training phase. By employing this algorithm, our agent can update the combat strategy by evaluating its surroundings, executing available maneuvers, and receiving feedback through reward signals. Experiments demonstrate that the models using TD3 and Expert-Augmented TD3 achieve 76% and 79% win rates, respectively, confirming the effectiveness and superiority of the proposed method.