Self-Play Meta-Reinforcement Learning in Multi-Agent Games

Abstract Interactions in multi-agent systems are often framed through the tools of game theory; however, in real-world scenarios, the structure and parameters of the underlying game faced by agents are frequently unknown or non-stationary. This presents a critical challenge: agents must rapidly infer the nature of their environment and adapt their strategies accordingly, even in the presence of multiple other agents. Meta-reinforcement learning (meta-RL) has demonstrated the ability to facilitate fast adaptation in tasks such as multi-armed bandits, Markov decision processes, and visual navigation. In this paper, we extend the application of meta-RL to multi-agent games. By training agents via self-play meta-reinforcement learning on diverse classes of normal-form games, parameterized by their payoff matrices and sampled from a distribution, we develop algorithms that are not only sample-efficient and robust to changes, but also capable of strategic generalization across distinct game-theoretic structures. Although it remains limited to a theoretical proof of concept, our approach bridges the gap between classical game-theoretic modeling and modern meta-learning techniques, with promising implications for adaptive behavior in dynamic multi-agent environments.