Nash Equilibrium in Multiplayer Graphical Games via Reinforcement Learning and Distributed Observers

Multiplayer game theory has been widely studied, with most existing research focusing on fully connected network structures. In contrast, multiplayer graphical games consider sparser communication topologies, making them more practical for large-scale systems. This article, based on a reinforcement learning (RL) method, investigates the problem of computing Nash equilibrium (NE) strategies in a class of multiplayer graphical games where the system is influenced by an external system. To estimate the unknown states of the external system, we propose a distributed adaptive observer and prove that its observation error asymptotically converges to zero. Furthermore, we derive a range of discount factor values that preserve system stability. To solve for the NE strategy, we develop an off-policy algorithm integrated with the distributed adaptive observer for policy evaluation. To enhance convergence speed, we introduce a distributed policy improvement mechanism, which ensures policy convergence to equilibrium while maintaining system stability. The effectiveness of the proposed algorithm is validated through simulations on a voltage synchronization system.