On Performance Improvement of Reinforcement Learning for Collision Avoidance in Autonomous Intersections

Reinforcement Learning (RL) is a promising technology to make autonomous vehicles (AV) a reality. The essential part of RL training is to derive an optimal policy to obtain the best performance in the intended environment. Here, the quality of the training and thus the effectiveness of the subsequent policy are dependent on a number of factors such as algorithm selection, environment definition, and parameter values. In this paper, we assert that the training process itself is an equally important determining factor for effective training. We conducted multiple training experiments using two RL algorithms in a multi-agent environment and a number of various configuration settings, and verified that optimizing the amount of input data based on its relevance significantly improves the training effectiveness in AVs navigating an unsignalized intersection. Our approach is specifically effective in a complex environment, where a significantly large amount of observable data hinders the effective policy convergence and quality of the training rather than helping it. By applying our design to identifying and eliminating data that are unimportant for the training objective, we reduced the number of observable inputs during the training process. As a result, we demonstrated a more effective training in a highly complex and dynamic environment such as AVs on the road. Furthermore, we achieved a higher success rate by adopting one policy per agent in a multi-agent environment as opposed to a single policy for all agents. Our results show 99% and 55∼78% reduction in collision in vehicle trajectories with and without turning at the intersection compared to baseline training performance, respectively. In addition to the quantitative metrics, we analyzed the qualitative behaviors of the learned policies to understand how strategic actions contributed to collision reduction. Our positive test result sheds light on a new approach to the RL training: data reduction in multi-policy training. We believe this newapproach is equally important to conventional approaches, such as algorithm selection and parameter tuning, toward effective RL training.