The Hamilton-Jacobi-Bellman Equation for Optimal Control in Multi-Agent Systems

Hamilton-Jacobi-Bellman (HJB) equation is fundamental to optimal control theory and is required for optimality using dynamic programming rules. We apply the HJB framework to a system with numerous agents who want to maximize their output objective while interacting with other agents in a common environment. In the cooperative and noncooperative cases, we formulate the coupled HJB equations governing the systems. Approximation techniques and a learning based approach to this challenge are presented to address key challenges such as the curse of dimensionality and the desire for decentralized solutions. We also study conditions under which Nash equilibria can be obtained from the HJB framework in differential games. The theoretical findings are validated with simulation results, and they demonstrate the application of the proposed methods in robotic coordination and autonomous vehicle systems.