Heterogeneous Multi-Agent Task Planning Method in Complex Marine Environment

To enable collaborative reconnaissance/strike/assessment of underwater time-sensitive targets by heterogeneous multi-agent systems, this study proposes a heterogeneous multi-agent collaborative decision-making method based on deep reinforcement learning (DRL). The method integrates two core frameworks: a heterogeneous multi-agent task allocation learning framework and a single-agent multi-task learning framework. The task allocation framework combines the Proximal Policy Optimization (PPO) algorithm with experience replay to train the Actor network, ensuring stable iterative updates of task allocation policies toward high-reward directions. Concurrently, the Critic network is trained using successor features and Temporal Difference (TD) methods, establishing a foundation for transfer learning. A region partitioning mechanism is introduced to construct a foundational knowledge base, enabling the transfer of sub-region knowledge acquired by multi-agents to target regions, thereby addressing complex underwater task allocation scenarios. The single-agent multi-task learning framework employs an experience replay pool with knowledge transfer attributes and policy distillation technology, allowing each agent to assimilate task-specific expertise from heterogeneous peers across diverse mission scenarios. This capability supports multi-task execution, including path planning, emergency obstacle avoidance, and trajectory tracking.