Multi-UAV Collaborative Reconnaissance Based on Multi Agent Deep Reinforcement Learning

In this paper, we explore a multi-agent reinforcement learning(MARL) method for the collaborative reconnaissance of unmanned aerial vehicles (UAVs). Most UAVs still use programmed control to perform reconnaissance tasks, and fully autonomous intelligent collaborative reconnaissance based on environmental conditions has not yet been achieved. To enable each drone to autonomously plan and cooperate with each other to complete dynamic target reconnaissance based on changing environmental conditions and target positions, we designed a multi-agent deep reinforcement learning algorithm for the collaborative reconnaissance of UAVs based on the multi-agent deep deterministic policy gradient (MADDPG). It adopts a centralized training and distributed execution strategy, where each drone uses its own Actor-Critic network to select and execute actions and then makes corresponding evaluations. Due to the many shortcomings of the original MADDPG algorithm, it is difficult to directly and effectively apply it to environments with a large number of intelligent agents and dynamic changes. Therefore, we have made several improvements to the original MADDPG algorithm, including the introduction of residual networks and parameter sharing, the implementation of entropy regularization exploration strategies, and the design of clear communication mechanisms. Simulation experiments have verified that our proposed improved algorithm can effectively achieve coordinated reconnaissance of UAVs and perform reconnaissance tasks without basic collisions. Compared with the original MADDPG algorithm, the simulation results show that the learning efficiency of our improved algorithm is significantly enhanced, and the reconnaissance effectiveness of UAVs is further improved.