Parameter Virtual Calibration Method for Vehicle Path Tracking Controller Based on the Deep Reinforcement Learning

<div class="section abstract"><div class="htmlview paragraph">Path tracking control, which is one of the most important foundations of autonomous driving, could help the vehicle to precisely and smoothly follow the preset path by actively adjusting the front wheel steering angle. Although there are a number of advanced control methods with simple structure and reliable robustness that could assist vehicles achieving path tracking, these controllers have many parameters to be calibrated, and there is a lack of guidance documents to help non-professional test site engineers quickly master calibration methods. Therefore, this paper proposes a parameter virtual calibration method based on the deep reinforcement learning, which provides an effective solution for parameter calibration of vehicle path tracking controller. Firstly, the vehicle trajectory tracking model is established through the kinematic relationship between the vehicle and the target path, combined with the Taylor series expansion linearization method. Next, a vehicle path tracking controller that considers the system following accuracy, steering smoothness and actuator characteristic constraints is established using Model Predictive Control (MPC) theory. At the same time, the parameters to be calibrated in the vehicle trajectory tracking controller are modeled as the agent action output of reinforcement learning, and the Twin Delayed Deep Deterministic Policy Gradient (TD3) technique is applied to optimally virtual calibrate them. Finally, a joint simulation platform is established by combining vehicle dynamics simulation software Carsim, scene simulation platform Prescan and MATLAB/Simulink. The experimental results show that the parameter virtual calibration method designed in this paper can help the vehicle to obtain better path tracking quality than the traditional manually calibrated MPC under various working conditions.</div></div>