Reinforcement Learning for Finite-Horizon H∞ Tracking Control of Unknown Discrete Linear Time-Varying System

This article considers the finite-horizon H<inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$_{\infty }$ </tex-math></inline-formula> tracking problem for a class of discrete linear time-varying systems. Two reinforcement learning (RL) methods—policy iteration (PI) and Q-learning—are proposed to solve this problem. The latter can obtain the H<inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$_{\infty }$ </tex-math></inline-formula> controller without system dynamics. In the field of RL control, most studies focus on infinite-horizon control and time-invariant systems, and few studies have investigated finite-horizon control or time-varying systems. In contrast to infinite-horizon H<inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$_{\infty }$ </tex-math></inline-formula> tracking control, finite-horizon H<inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$_{\infty }$ </tex-math></inline-formula> tracking control involves a time-varying value function. While this introduces challenges, it empowers the algorithm to effectively handle time-varying problems. Within the finite-horizon framework, the value function is bounded, allowing the removal of the discount factor, thereby enhancing control performance. Additionally, there is no longer a need for an admissible control law for initialization, providing the proposed algorithms with the combined advantages of both PI and value iteration (VI). Two simulation examples are used to verify the effectiveness of the proposed algorithms.