Online Multi-player Resource-Sharing Games with Bandit Feedback

Abstract This paper considers an online multi-player resource-sharing game with bandit feedback. Multiple players choose from a finite collection of resources in a time slotted system. In each time slot, each resource brings a random reward that is equally divided among the players who choose it. The reward vector is independent and identically distributed over the time slots. The statistics of the reward vector are unknown to the players. During each time slot, for each resource chosen by the first player, they receive as feedback the reward of the resource and the number of players who chose it, after the choice is made. We develop a novel Upper Confidence Bound (UCB) algorithm that learns the mean rewards using the feedback and maximizes the worst-case time-average expected reward of the first player. The algorithm gets within $$\mathcal {O}(\log (T)/\sqrt{T})$$ of optimality within T time slots. The simulations depict fast convergence of the learnt policy in comparison to the worst-case optimal policy.