MHRL-RD: Multi-Agent Hierarchical Reinforcement Learning Based on Roles Distribution

In actual traffic environments, there are multiple obstacles to using multi-agent reinforcement learning algorithms to achieve vehicle cluster collaboration tasks, including limited sensing range, difficulty in updating the overall state space in time, excessive policy space, and difficulty in training convergence when the number of agents increases. Because of the above considerations, this paper takes the strongly correlated multi-agent formation collaboration task based on limited communication and sensing range as the background, obtains updated map information through distributed information collection, and uses a market-based or game-based method to allocate a certain role to each agent. Then, it allocates different subtasks to different roles and updates their policy networks, respectively, so as to decompose and streamline the overall too large policy network and improve training efficiency. In this process, a hierarchical collaborative strategy is used to complete actual training, to reach final overall task completion and real-time requirements.