An attention-based dual-evaluation learning architecture for collaborative target encirclement

For multi-agent collaborative encirclement tasks, this paper addresses the challenges encountered in traditional multi-agent deep reinforcement learning (MADRL) methods, particularly the constraints posed by credit assignment and low learning efficiency issues. To tackle these problems, a modified MADRL algorithm is developed specifically for multi-agent systems, integrating a dual critic network and a multi-head attention mechanism. The dual critic network comprises a local critic network and a global critic network, with each network independently assessing the effectiveness of actions for the individual agent and the team, respectively, to achieve a balanced benefit. The multi-head attention mechanism is incorporated into the global critic network, and the attention weight of each agent is dynamically allocated to ensure that key information is effectively and precisely captured and processed, thereby alleviating the influence of local features and biases, and addressing the issue of low learning efficiency. A segmented reward function is designed to analyze the advantages and disadvantages of agent actions by calculating the distance between each agent and the escape target, and then comprehensively evaluating the behavioral value of all agents, so as to promote cooperative behavior. Numerical simulations demonstrate that the proposed system yields substantial improvements in performance metrics, including encirclement speed, travel distance, and encirclement success rate.