How to Tame a Multi-Armed Bandit? A Multi-Criteria Decision Analysis Approach to Bayesian Adaptive Exploration in Non-Stationary Environments

The multi-armed bandit problem in non-stationary environments requires the development of methods that effectively balance the exploration of new options with the exploitation of those already identified as beneficial, while also dynamically adapting to changes in reward distributions. In this study, we propose and analyze the Bayesian Adaptive Epsilon-Greedy (BAEG) algorithm, specifically designed to address non-stationarity. The primary objective of the study is to evaluate the effectiveness of BAEG by comparing its performance against other widely used adaptive strategies. The algorithm combines the classical assumptions of the epsilon-greedy approach with Bayesian updates of beliefs about reward distributions, enabling more flexible responses to changing conditions. Simulation experiments conducted across a series of tasks with non-stationary reward distributions demonstrate that BAEG not only matches but often outperforms other adaptive methods, particularly in scenarios characterized by frequent and abrupt changes. The results highlight the potential of Bayesian approaches for adaptive management of exploration and exploitation in multi-armed bandit problems and provide a foundation for future research on generalized variants of such algorithms.