A Data-Driven Adaptive Optimal Control for Continuous-Time Nonlinear Systems with Unknown Dynamics from Unstable Initial Policy

This paper introduces a novel model-free adaptive optimal control approach for continuous-time nonlinear systems, which addresses several limitations of existing Value Iteration (VI) methodologies—particularly their reliance on prior knowledge of system dynamics, constraints associated with discrete-time systems, and the significant dependence of Policy Iteration (PI) on initial stable control strategies. A comprehensive data-driven continuous-time VI framework is developed to eliminate the traditional reliance on initial admissible control policies. Furthermore, a rigorous proof of convergence and closed-loop stability is provided. The key innovation of the proposed approach lies in its use of real-time state and input data streams to simultaneously identify dynamic model parameters and optimize the control policy. Experiments validate the proposed algorithm.