Exploration of deep reinforcement learning method for cooperative control of connected automated vehicles at a roundabout

Abstract This study explores a deep reinforcement learning mechanism to obtain cooperative driving control of connected autonomous vehicles at a roundabout. A roundabout is one of the intersections without signal control, and we focus on it because high-level cooperation is required to merge and branch off at each junction. This study introduces three new mechanisms to the previous model that enable learning cooperative control: ( $$\textrm{i}$$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mtext>i</mml:mtext> </mml:math> ) curriculum learning that decreases the new vehicle departure interval during the training so that the number of vehicles in a roundabout gradually increases, ( $$\textrm{ii}$$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mtext>ii</mml:mtext> </mml:math> ) utilization of information about the vehicle’s destination, and ( $$\textrm{iii}$$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mtext>iii</mml:mtext> </mml:math> ) an additional penalty for approaching walls and other vehicles for safe driving. We conducted simulation experiments to investigate the effectiveness of the proposed methods. The experimental results showed that the proposed methods enable the acquisition of cooperative vehicle control that can arrive at each destination while reducing the collision rate.