Online adaptive surrogates for the value function of high-dimensional nonlinear optimal control problems

We introduce a strategy that generates an adaptive surrogate of the value function of high-dimensional nonlinear optimal control problems. It exploits the relevant operating domain online on which the resulting surrogate satisfies the Hamilton–Jacobi–Bellman (HJB) equation up to a given threshold. The approximate value function is based on Hermite kernel regression, where the data stems from open-loop control of reduced-order optimal control problems. As a measure of accuracy, the full-order HJB residual, known as the Bellman error, is used to determine whether the current Hermite kernel surrogate is sufficient or further training is required. In addition, the reduced-order model can also be improved using the full-order data if the same HJB-based error indicator suggests that the current reduced system is not accurate enough. Numerical experiments support the effectiveness of the new scheme.