Consistent Action for Stable Training in Reinforcement Learning–based Gain Tuning of Linear Feedback Controller

Controlling nonlinear systems with linear feedback controller after linearization is a widely used method. This paper proposes a new method to efficiently train a reinforcement learning agent to select the control gain of the linear feedback controller. The proposed method involves the agent selecting a consistent gain by choosing between a new and an old gain. Since the linear feedback controller can operate with this consistent gain, the agent does not need to select the optimal gain, thus simplifying the training process. Numerical simulations were performed by applying the proposed method to a longitudinal model of a nonlinear missile with a 3-loop autopilot. The proposed method enables training with a wide range of gains and enhances the reference tracking performance compared to classical gain scheduling.