CVaR-based Thompson Sampling for Factored Multi-agent Multi-Armed Risk-aware Bandit

Risk-aware online learning and bandit algorithms have been successfully applied in scenarios where optimizing the expected reward is not sufficient to achieve desired outcomes due to the presence of uncertainty. However, distributed decision-making under uncertainty in multi-agent systems has become increasingly important for addressing real-world tasks. Our research focuses on developing a distributed risk-aware decision-making approach that optimizes a shared objective in a network of loosely coupled agents. Specifically, we address the problem of regret minimization in the multi-agent multi-armed risk-aware bandit (MAMARAB) framework, where each arm follows a Gaussian distribution with bounded variances and is evaluated using Conditional Value at Risk (CVaR). To tackle this problem, we propose a novel CVaR-based multi-agent Thompson sampling (CVaR-MATS) algorithm, which selects actions based on the CVaR measure to balance exploration and exploitation while taking into account the risk associated with each arm. Moreover, to provide theoretical guarantees, we derive a regret bound for the CVaR-MATS algorithm that is sub-linear in time and exhibits low-order polynomial growth concerning the highest number of actions in sparse coordination graphs.