Self-organized Swarm Reinforcement Learning under Spontaneous Communications

Reinforcement learning refers to the ability of an agent to learn from its sequential interactions with the environment to find the optimal behavior that maximizes a particular reward. It is a crucial building block of autonomous intelligent systems. In recent years, there has been increasing attention on techniques for coordinating large numbers of autonomous agents to perform tasks cooperatively. A potentially attractive design strategy is to learn from naturally existing swarm intelligence systems and employ self-organized control paradigms. In this paper, we explore the viability of a self-organized reinforcement learning strategy and aims to solve the multi-agent multi-armed bandit problem for anonymous agents with only spontaneous communications between them. The proposed swarm reinforcement learning approach is compared against the individual implementation of the classical UCB1 algorithm, and the regret generated under different algorithmic and environmental parameter settings are measured and analyzed. The proposed approach is shown to be able to outperform the baseline and also scalable to large decision spaces and swarm sizes.