Optimal Adaptive Control of Linear Stochastic Systems With Quadratic Cost Function

This article focuses on the adaptive linear quadratic Gaussian control problem, where both the state matrix <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$A$</tex-math></inline-formula> and the control gain matrix <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$B$</tex-math></inline-formula> are unknown. We only assume that <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$(A, B)$</tex-math></inline-formula> is stabilizable and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$(A, Q^{1/2})$</tex-math></inline-formula> is detectable, where <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$Q$</tex-math></inline-formula> is the weighting matrix of the state in the quadratic cost function. This significantly weakens the classic assumptions used in the literature. To design an optimal adaptive control, a weighted least squares algorithm is modified by using random regularization method, which can ensure uniform stabilizability and uniform detectability of the family of estimated models. At the same time, a diminishing excitation is incorporated into the design of the proposed adaptive control to guarantee strong consistency of the desired components of the estimates. Finally, although some components of the estimates may not converge to the true values, it is still demonstrated that a certainty equivalence control with diminishing excitation remains optimal for an ergodic quadratic cost function.