Data-Driven nonlinear optimal control via feedback-based extremum seeking

In this paper, we propose EXPronto, a data-driven approach for nonlinear optimal control problems with unknown dynamics. Our method combines the numerical stability benefits of feedback-based optimal control methods with the computational efficiency and minimal information requirements of extremum-seeking algorithms. More in detail, in our approach, the system is run in closed loop, thanks to an available feedback policy, to collect values of the cost function along a perturbed trajectory estimate. The gathered cost values are then used to update the trajectory estimate by relying on a mechanism based on extremum seeking. We theoretically guarantee local convergence of EXPronto in an arbitrarily small neighborhood of an isolated optimal trajectory. Specifically, by using system theory tools based on averaging theory, combined with ideas from nonlinear optimal control, we show local practical exponential stability properties for the algorithm evolution. We numerically test the effectiveness of our method.