Safe Autonomous Driving via Deep Reinforcement Learning and Model Predictive Control

In this paper, the following lane-changing decision problems of autonomous vehicles on highways are studied. An autonomous vehicle driving on a highway with several cars around it. For the autonomous vehicle, the dynamic model of the vehicle is discussed, a two-layer controller combining deep reinforcement learning (DRL) and model predictive control (MPC) is proposed. A high-level strategy for the discrete actions of the ego vehicle is designed using a Deep Q-Network (DQN), which can compress the solution space of low-level controller and an MPC-based low-level controller is implemented to enhance efficiency and safety. A predictive neural network based on Convolutional Long Short-Term Memory (ConvLSTM) is applied to assign early negative rewards for actions that could lead to potential future collisions. The results are compared with the control group, which used PID control as its low-level controller. The effectiveness of the proposed controller is verified on an open autonomous driving simulation platform.