Genetic Action Set Generation for Monte Carlo Tree Search in Six-Degree-of-Freedom Multi-Agent Overmatch

Teams of agents offer the potential to defeat higher-performance adversaries by employing intelligent coordination. This work examines how Monte Carlo tree search (MCTS) can be utilized for multi-agent, six-degree-of-freedom (6-DoF) engagements. This work introduces a novel approach to action space selection for known MCTS algorithm frameworks. This enables intelligent coordination for 6-DoF multi-agent adversarial game environments with continuous state and action spaces. Our framework leverages a genetic algorithm to determine subsets of algorithmically generated actions to overcome disadvantaged situations. We quantify the benefits of our proposed approach by examining multi-agent aerial engagements with F-16s and P-51Ds. We compare our approach to uncoordinated pure pursuit and reinforcement learning agents. Our results illustrate that the proposed genome-action-informed MCTS substantially increases the win rate compared to pure pursuit and baseline MCTS. These results hold across both aircraft platforms. In addition, we show how our algorithm can achieve high win rates even when operating at a large maneuverability or speed disadvantage. These results illustrate the benefits of MCTS for coordinating 6-DoF engagements and highlight the benefits of intelligent action space selection.