Biased random masked attention MAPPO algorithm for zero-shot scale generalization of multi-UAV air combat

Abstract With the wide application of uncrewed aerial vehicles (UAVs) in the military, multi-UAV cooperative air combat will become an important form of air combat. For the problem of multi-UAV swarm scale generalization, this paper proposes a biased random masked attention MAPPO (BRMA-MAPPO) algorithm, which achieves zero-shot scale generalization of multi-UAV swarms. Specifically, an entity observation encoder based on multi-head attention is introduced to address the variable number of multi-UAV swarms. Meanwhile, a biased random mask is proposed to enhance the performance at zero-shot scale generalization. The mask vector is generated by a mask vector generator network and applied to the attention matrix, masking non-essential entity information, thereby improving the generalization performance. The loss function of the mask vector generator network is constructed based on the information bottleneck theory, and the gradient of the mask network is derived. The simulation results show that the entity observation encoder proposed in this paper can help focus on key battlefield entity information and improve the training effect. The biased random mask proposed in this paper can mask non-important entity information, maintain focus on key entities when generalizing to larger scenarios, and improve generalization effects. Compared with the existing zero-shot generalization multi-agent reinforcement learning algorithms, BRMA-MAPPO can significantly improve the generalization performance. In the 8V8 and 10V10 zero-shot generalization scenarios, the method proposed in this paper outperformed state-of-the-art algorithms by 12% and 15% in terms of winning rate, respectively.