On-Policy Data-Driven Linear Quadratic Optimal Control of SISO Systems via Model Reference Adaptive Reinforcement Learning

In this paper, we address the problem of on-policy data-driven linear quadratic optimal control for continuous-time single-input single-output systems. Assuming that the plant is minimum phase and has relative degree one, we propose model reference adaptive reinforcement learning – an approach with theoretical guarantees that combines learning and model reference adaptive control. The developed algorithm features an adaptive output-feedback controller that tracks a parameter-varying reference model, whose behavior is shaped by a discrete-time optimizer. For the resulting hybrid closed-loop system, we establish semi-global boundedness of the solutions and show that, under persistency of excitation induced by a dither signal, the applied policy converges to the optimal one.