Variational Stochastic Games

The Control as Inference (CAI) framework has successfully transformed single-agent reinforcement learning (RL) by reframing control tasks as probabilistic inference problems.However, the extension of CAI to multi-agent, general-sum stochastic games (SGs) remains underexplored, particularly in decentralized settings where agents operate independently without centralized coordination.In this paper, we propose a novel variational inference framework tailored to decentralized multi-agent systems.Our framework addresses the challenges posed by non-stationarity and unaligned agent objectives, proving that the resulting policies form an -Nash equilibrium.Additionally, we demonstrate theoretical convergence guarantees for the proposed decentralized algorithms.Leveraging this framework, we instantiate multiple algorithms to solve for Nash equilibrium, mean-field Nash equilibrium, and correlated equilibrium, with rigorous theoretical convergence analysis.