Coordinating Autonomous Vehicles Using Reinforcement Learning

Vehicle coordination enables autonomous agents traversing the same complex environment to navigate it safely and efficiently. We evaluate three primary approaches to multi-agent coordination in simple vehicles and compare their performance. The three approaches, which are all based on reinforcement learning, are as follows: a centralized model, a decentralized model without communication, and a decentralized model with communication. We evaluate the three schemes via training and simulations using the OpenAI Gym framework in a closed-loop environment, where two agents are traveling in opposite directions (clockwise or counterclockwise), and must coordinate to avoid collisions while maximizing completed laps before colliding or reaching the cumulative time-step limit. We hypothesize that the fully centralized model would result in the best overall performance due to full action-state communication, while the decentralized model with communication outperforms those without. The results of the study partially support this conjecture. The duration of the episode under the fully centralized approach exceeds that of both decentralized models, resulting in the highest long-term accumulated reward. The decentralized model with communication performs better than the one without due to longer episode duration, a higher number of laps, and greater long-term reward accumulation. The decentralized model excels in scalability, underscoring the challenge of balancing coordination with computational efficiency. Centralized models exhibit the highest long-term rewards, while decentralized models excel in parallel processing and scalability.