Action-Integrated QAttn: Introducing Action Values into Value Decomposition for Effective Cooperation among Heterogenous Agents

Multi-agent deep reinforcement learning (MARL) approaches, such as VDN, QMIX, and Qatten, decompose the global value $Q_{t o t}$ into individual action values $Q_{i}$, promoting cooperation among agents. However, these methods typically assume that agents perform homogeneous tasks or have predefined roles. This paper addresses environments where agents must autonomously allocate heterogeneous tasks, such as in the preypredator problem, without explicitly defined roles. To tackle this challenge, we propose a novel method that uses action values as agent features, combined with attention mechanisms. Our approach aims to enable effective learning of task allocation strategies in environments where roles are not explicitly assigned.