PID Control by Safe Reinforcement Learning for Vehicle Lateral Control

This work is an extension of the previous work by focusing on online tuning of the Proportional Integral Derivative (PID) controller using Reinforcement Learning (RL). Unlike the offline tuning approach previously explored, online tuning allows for real-time adaptive adjustments, ensuring stability and effective RL behavior in real-world scenarios, especially during training. The linear-quadratic Gaussian (LQG) cost function is used as a reward function to optimize the PID parameters. The cost combines tracking error reduction and actuator effort limitation, which constrains the energy or force used by the system's actuators to prevent saturation. Two learning agent performances are explored: the simple Deep Deterministic Policy Gradient (DDPG), and more advanced agent Twin Delayed Deep Deterministic (TD3). The learning procedure checks the closed-loop stability before the parameters update. DDPG and TD3 are compared in terms of performance to MAT LAB Simulink control parameter tuner. This work contributes to the methodology and application of reinforcement learning.