From Board Games to Esports: Multi-agent Deep Reinforcement Learning Across Game Types

Multi-agent deep reinforcement learning has made significant progress in game applications. However, it is still unknown that why some algorithms work well in a specific game but not in others. This paper proposes a classification-based survey covering five game categories: perfect-information board games, imperfect-information card games, real-time strategy games, sports games, and multiplayer online competitive games. Moreover, it analyzes its key features, technical challenges, and representative algorithms for these kinds of game categories .Board games use Monte Carlo Tree Search which is accompanied by self-play; card games use counterfactual reasoning for information that is imperfect; real-time strategy games use centralized critics, policy optimization, and value decomposition; sports games have competitive self-play function in continuous action environments; multiplayer online games use methods that use large-scale self-play and population learning to make systems more scalable. Comparison between different types shows shared challenges including non-stationarity, credit assignment, and scalability, and introduce three future directions to lead algorithm design between different game types.