Multiplayer hierarchical decision‐making for discrete‐time nonlinear networks of service via value iteration adaptive dynamic programming

Abstract In this article, the multiplayer hierarchical decision‐making problem for discrete‐time nonlinear networks of service is studied from the perspective of Nash–Stackelberg–Nash games. A novel two‐level value iteration adaptive dynamic programming algorithm is developed to solve the coupled nonlinear Hamilton–Jacobi–Bellman equations associated with the game problem, which neither requires the system drift dynamics to be known as a priori nor requests the initial strategies to be admissible. Moreover, both the value function sequences and the strategy sequences generated by the algorithm converge to their theoretical optimality. An implementation framework for the algorithm is constructed by leveraging the regularized least‐squares method, and a convergence criterion that guarantees the admissibility of the approximated strategies is also proposed. The effectiveness of the proposed algorithm and implementation framework are finally demonstrated through two numerical simulation examples.