Mapping Tracking Control to Cascading Optimization in Discrete Strict-Feedback Systems: A Hierarchical Learning Approach

This article introduces a hierarchical learning (HL) framework for discrete-time (DT) systems with strict feedback structures to enable tracking control. Unlike the backstepping approach, our method creates dynamically adjustable virtual targets (VTs) for state variables at each layer, forming a cascading optimization structure. This innovative framework enables each layer to learn by approximating the solution to the DT Hamilton-Jacobi-Bellman (HJB) equation, thereby facilitating inter-layer self-optimization and directing the modification of adjacent VTs. To tackle the noncausal problem, we implement an iterative predictive learning framework that maps the current measurable state and known reference trajectories to VTs. This process allows the VTs to gradually align with the optimal trajectory during policy evaluation and update, achieving indirect tracking of state variables toward the desired targets. Additionally, the action network is transformed into a tracking network, incorporating future tracking errors to optimize its weights. This approach reduces tracking costs in the subsequent policy update while improving tracking performance. Rigorous convergence analysis and numerical simulations confirm the effectiveness of our method, highlighting its considerable potential in adaptive control.