Discrete-time optimal tracking control for unknown nonlinear systems via a novel phase-error-based Q -learning method

In this paper, a novel phase-error-based Q-learning algorithm is proposed to solve the optimal tracking control problems for unknown discrete-time nonlinear systems. For the first time, phase errors between the system state and the desired trajectory are incorporated into the performance index, which rigorously guarantees the asymptotic convergence of the tracking error to zero. Building on this index, a value-iteration-based Q-learning scheme is developed to derive the optimal tracking controller. The convergence of the iterative Q-function is rigorously proven, and a termination condition ensuring admissibility of the resulting policy is provided. Unlike existing adaptive dynamic programming (ADP) approaches, our method avoids pre-computed desired controls or initial admissible solutions. It enables optimal tracking control without requiring system dynamics or pre-established effective control strategies. Finally, two neural networks implement the algorithm and simulations complement the theoretical discussions.