Design and implementation of cooperative multi-agent based on MAPPO

In the past few years, with the continuous development of deep reinforcement learning, the application of deep reinforcement has been set off within the realms of self-driving vehicle technology, game games, and natural language processing, and the proximal policy optimization algorithm has emerged as a focal point in the sphere of deep reinforcement learning research. In this paper, the Proximal Policy Optimization (PPO) algorithm and the Multi-Agent Proximal Policy Optimization (MAPPO) algorithm are studied, and a cooperative multi-agent system is designed and implemented based on the MAPPO algorithm in the field of StarCraft game, which is compared with the popular Q-Value Mixing (QMIX) reinforcement learning algorithm in recent years, and proposes a direction for subsequent system improvement. In the proximal policy optimization algorithm, the traditional policy gradient algorithm, natural policy gradient algorithm, trust domain policy optimization algorithm, proximal policy optimization algorithm, and independent proximal policy optimization algorithm are studied. In the multi-agent proximal policy optimization algorithm, the centralized training decentralized execution architecture, athlete critic method, value standardization, experience playback, and GAE (GAE) adopted by the MAPPO algorithm are studied. Designing and implementing a cooperative multi-agent system, introducing the StarCraft environmental campaign. Regarding the specifics of the training process, the policy gradient algorithm is used to update the policy network, the time series difference algorithm is improved, and the generalized advantage estimation is used to update the value network. Compared with the QMIX learning algorithm, the model trained by the MAPPO algorithm has the advantages of faster convergence and better stability.