The unmanned vehicle on-ramp merging model based on AM-MAPPO algorithm

In response to the issues of low merging success rates and poor safety in the on-ramp merging scenario within autonomous driving, we propose an on-ramp merging model for unmanned vehicles based on the Multi-Agent Proximal Policy Optimization (MAPPO) algorithm. Firstly, we introduce an Action-Mask (AM) to prevent the sampling of invalid actions during merging, thus enhancing safety by ensuring only valid actions are considered. Secondly, we incorporate noise advantage values to encourage unmanned vehicles to thoroughly explore the environment and avoid being trapped in local optimal solutions. Experimental results demonstrate that the AM-MAPPO algorithm model improves both safety and traffic efficiency.