Almost Surely $\sqrt{T}$ Regret for Adaptive LQR

The linear-quadratic regulation (LQR) problem with unknown system parameters has been widely studied, but it has remained unclear whether <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\tilde{ \mathcal {O}}(\sqrt{T})$</tex-math></inline-formula> regret, which is the best known dependence on time, can be achieved almost surely. In this article, we propose an adaptive LQR controller with almost surely <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\tilde{ \mathcal {O}}(\sqrt{T})$</tex-math></inline-formula> regret upper bound. The controller features a circuit-breaking mechanism, which falls back to a known stabilizing controller when the input signal or control gain is large, thereby circumventing potential safety breach and ensuring the convergence of the system parameter estimate. Meanwhile, the circuit-breaking is shown to be triggered only finitely often, and hence has a negligible effect on the asymptotic performance of the controller. The proposed controller is also validated via simulation on Tennessee Eastman process (TEP), a commonly used industrial process example.