Gradient-Based Algorithms With Intermediate Observations in Static and Differential Games

In two-player static and differential games, strategic players often use available or delayed information about the other player’s decisions and solve an optimization or optimal control problem to determine their strategic choices. Without this information, the player’s ability to determine its optimal decisions becomes problematic. In this paper, we propose an approach in which each player implements an iterative discrete-time gradient-based algorithm that relies only on intermediate either current or prior observations about the other player’s actions. We explore the implementation of such gradient play algorithms in the case of non-zero-sum static games and in the more complex case of differential games. We discuss the properties of these algorithms with heterogeneous stepsizes and derive explicit necessary and sufficient conditions on the game parameters in the objective functions and stepsizes that guarantee convergence to the Nash equilibrium in static games with quadratic objective functions. Examples in both static and differential games are presented to illustrate the results.