MAPPO and Hungarian Algorithm Based Collaborative Decision-Making for UAV Air Combat

This paper investigates the problem of coordinating <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$N$</tex> unmanned aerial vehicles (UAVs) to shoot down <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$M$</tex> targets (N ≥M) through maneuvering decisions in a three-dimensional space without boundary constraints. The focus is on ensuring that all agents avoid collisions while employing collaborative strategies to efficiently neutralize the targets. A Multi-Agent Proximal Policy Optimization combined with the Hungarian algorithm (MAPPO-H) is designed, integrating multi-agent reinforcement learning with a target allocation method. The Hungarian algorithm is utilized to address the target allocation problem, while MAPPO is utilized to implement the maneuver decision-making strategy. Firstly, we establish the aircraft motion model and define conditions for achieving victory in battle. Secondly, we frame the autonomous maneuver decision-making as a Markov Decision Process (MDP), delineating the state space, action space, and reward function using a target allocation algorithm rooted in the Hungarian method. We finally give a numerical simulation to verify the proposed algorithm and the conclusion that all agents can avoid collisions and successfully shoot down all targets.