Automatic tuning of controller parameters based on reinforcement learning with applications in intelligent vehicle path tracking

Controller parameters tuning is a tedious work for engineers since it always requires a number of repetitive trials. In this paper, we focus on the automatic tuning of controller parameters, aimed at eliminating the requirement of expertise and reducing the trials required. We extend the original PILCO, a model-based reinforcement learning framework known for data efficiency, to automatically tune complex nonlinear controllers directly from real-world trial data. The key improvement of our method is that we calculate the probability distribution of control through Monte Carlo sampling rather than an analytical method, which avoids the problem of the expression of the control function relative to the state in the integrand function being too complex to analytically calculate the distribution of control in nonlinear controllers, greatly improving the flexibility of the original PILCO in nonlinear controllers. To verify its effectiveness, the proposed method has been applied to tune three path tracking controllers for an intelligent vehicle, including a PI, pure pursuit (PP) and enhanced Stanley controllers. Experiment results show that the proposed method can achieve automatic parameter tuning in a few trials, and controllers tuning by improved-PILCO exhibit high tracking accuracy in lane change of 40 km/h and intricate path. The proposed method is hopeful to accelerate the development of control module for intelligent vehicles.