The Innate Curiosity in the Multi-Agent Transformer

Curiosity is a cognitive mechanism that drives one's intrinsic need to understand the unknown. This intrinsic drive is responsible for guiding the acquisition of knowledge about novel stimuli. Curiosity, akin to thirst and hunger, is considered an evolved motivational mechanism promoting self-beneficial actions. In the context of Multi-Agent Reinforcement Learning (MARL), curiosity encourages exploration by capturing the novelty of an environmental state as an intrinsic reward signal. For cooperative MARL tasks, the Multi-Agent-Transformer (MAT) is one of the state-of-the-art models. However, its performance on sparse reward tasks requiring collaboration is uncertain. This paper explores MAT's performance on the grid-world environment Multi-Robot Warehouse. We integrated an Intrinsic Curiosity Module (ICM) for exploration and our results suggests that MAT does not need ICM to learn on sparse environments.