Adaptive Game Mechanics: Leveraging Multi-Armed Bandits for Dynamic Difficulty Adjustment

Dynamic Difficulty Adjustment (DDA) is a crucial task in video game design to select the appropriate difficulty level to maintain player engagement. Recent studies have highlighted the importance of developing adaptive systems that balance challenge and enjoyment. This paper explores the application of Multi-Armed Bandit algorithms to DDA, providing a computationally efficient and explainable solution to real-time difficulty adjustment. A numerical game engine was built with a damage calculator and a linear attribute generator for players and monsters. The study tested four Multi-Armed Bandits (MAB) algorithms: epsilon-greedy, Upper Confidence Bound, Asymptotically Optimal Upper Confidence Bound, and Thompson Sampling within the DDA framework. Results showed that all MAB algorithms successfully identified the optimal difficulty within 100 trials, and on average, players won their battles in more than 150 rounds. These results demonstrate the strong performance of MAB algorithms in dynamic difficulty adjustment and indicate the potential for implementing them in more complex gaming environments. The findings suggest that MAB could significantly enhance the adaptability and efficiency of DDA systems in future game development.