A Novel Reinforcement Q-Learning Control Approach for Lateral Vehicle Dynamics

This paper introduces a novel reinforcement Q learning algorithm equipped with an actor-critic neural network (NN) structure to effectively control the lateral dynamics of an autonomous vehicle (AV), even when system dynamics are entirely unknown. To address the challenges posed by traditional persistence of excitation conditions, an experience replay technique is implemented, leveraging past data to train the adaptive critic law. A nonlinear lateral vehicle dynamic model is established, considering steering angle as the input and road curvature as the disturbance. Subsequently, a linearized model is derived using the Euler technique. The Q-function is formulated using the Hamilton-Jacobi-Bellman (HJB) equation, and its parameters are estimated through integral reinforcement learning (IRL) and an online actor-critic NN, ensuring closed-loop stability. Finally, the efficacy of the proposed approach is demonstrated through a simulation example conducted on an AV.