A Deep Reinforcement Learning Approach for Controlling Autonomous Vehicles in Lane-Free Roundabouts

Lane-free traffic is a new concept proposed for the era of connected and automated vehicles (CAVs). In this system, vehicles are no longer restricted to traditional lanes, and any lateral location within the entire road boundaries is considered for navigation. In the current lane-based traffic system, roundabouts, characterized by wide lanes or no clear lane markings, allow vehicles to have more lateral movement freedom and thus offer an ideal setting to investigate how CAVs should behave in lane-free conditions. This study introduces a new approach to controlling CAVs in a lane-free urban environment using Deep Reinforcement Learning (DRL). This constitutes the first time DRL has been applied to help intelligent vehicles drive through urban roundabouts without the constraints of traditional lanes. By allowing vehicles to use the entire road space, the model aims to provide a comfortable and collision-free driving experience, enabling vehicles to maintain desired speeds. Our methodology involves developing a Deep Deterministic Policy Gradient (DDPG) based control strategy that enables CAVs to make dynamic, real-time decisions for efficient navigation, merging, and exiting. To test this approach, we simulated a real-world roundabout, already deploying a lane-free design. We applied our model to all CAVs driving under various traffic patterns in that environment. We also compared its performance to a two-dimensional control strategy based on the self-driven particle model for mixed traffic. The findings indicate that our DRL-employing autonomous vehicles are able to learn smooth driving policies and achieve target speeds, in addition to avoiding collisions and ensuring a comfortable experience for passengers.