A Physics‐Informed Learning Framework to Solve the Infinite‐Horizon Optimal Control Problem

ABSTRACT We propose a physics‐informed neural networks (PINNs) framework to solve the infinite‐horizon optimal control problem of nonlinear systems. In particular, since PINNs are generally able to solve a class of partial differential equations (PDEs), they can be employed to learn the value function of the infinite‐horizon optimal control problem via solving the associated steady‐state Hamilton‐Jacobi‐Bellman (HJB) equation. However, an issue here is that the steady‐state HJB equation generally yields multiple solutions; hence, if PINNs are directly employed on it, they may end up approximating a solution different from the optimal value function of the problem. We tackle this by instead applying PINNs to a finite‐horizon variant of the steady‐state HJB with a unique solution that uniformly approximates the optimal value function as the horizon increases. An algorithm to verify if the chosen horizon is large enough is also given, as well as a method to extend it—with reduced computations and robustness to approximation errors—in case it is not. Unlike many existing methods, the proposed technique works well with nonpolynomial basis functions, does not require prior knowledge of a stabilizing controller and does not perform iterative policy evaluations. Simulations are performed, which verify and clarify theoretical findings.