Multi-agent path tracking control based on Attention-enhanced TD3 reinforcement learning

To tackle the shortcomings of traditional centralized control methods in dealing with high-dimensional state spaces and local observation constraints, we propose a cooperative control algorithm based on the Twin Delayed Deep Deterministic Policy Gradient (TD3) reinforcement learning framework integrated with an attention mechanism. The proposed method constructs a distributed Actor-Critic architecture and introduces a multi-head attention mechanism to enable key information extraction and dynamic association modeling across agents. At the state representation level, a joint observation encoder based on spatiotemporal feature fusion is designed to adaptively fuse heterogeneous sensor data through multi-head attention weight allocation. The control strategy network adopts the dual critic structure of the TD3 algorithm and the delayed update mechanism, effectively alleviating the overestimation issue. Experimental results demonstrate that our proposed method outperforms benchmark algorithms, achieving a significant reward increase of over 45% and reducing position synchronization error to less than 0.02 meters, thereby validating the effectiveness of the multi-attention mechanism in multi-agent cooperative tasks and providing a new solution for industrial multi-arm control.