Enhanced exploration for multi-UAV cooperative roundup: An I2C-MATD3 reinforcement learning framework

With the increasing maturity of multi-UAV technology and its broad applications in scenarios such as UAV roundup tasks, this paper proposes a novel approach to enhance interception efficiency and system robustness by addressing insufficient historical data utilization and inadequate environmental exploration. The multi-UAV roundup problem is formulated as a Markov Decision Process (MDP), and an Improved Cross-Entropy Method with Intrinsic Curiosity-enhanced Multi-Agent Twin Delayed Deep Deterministic Policy Gradient (I2C-MATD3) is designed. Specifically, an Improved Cross-Entropy Method (ICEM) based on global elite samples rapidly optimizes training strategies while generating extensive experience for a Multi-Agent Twin Delayed Deep Deterministic Policy Gradient algorithm augmented with intrinsic curiosity rewards (IC-MATD3). In turn, IC-MATD3 guides the optimization direction of ICEM, enabling a synergistic interaction that facilitates effective historical data exploitation and proactive environmental exploration for UAV agents to accomplish roundup tasks. Experiments in complex scenarios demonstrate that the proposed algorithm achieves superior training efficiency and convergence performance compared to state-of-the-art multi-agent reinforcement learning (MARL) methods. Robustness tests and ablation experiments further validate its enhanced generalizability and robustness.