Permutation Equivariant Deep Reinforcement Learning for Multi-Armed Bandit

Permutation equivariance (PE) is a property widely present in mathematics and machine learning. Classic deep reinforcement learning (DRL) algorithms, such as Deep Q-Network (DQN), require thoroughly exploring the state space to achieve optimal performance. For a PE problem such as the Multi-Armed Bandit (MAB) problem, the PE property helps reduce the space that needs to be explored. This paper proposes PEDQN, a PE DRL framework based on DQN by applying a PE neural network structure. Our MAB experiments show that PEDQN has clear advantages compared to DQN with a fully connected network and achieves the same or better performance than UCB1 when tested in the same environment as the training.