Two-Stage Multi-UAV Path Planning Based on MAPPO

The adaptive path planning of multi-UAV in dynamic environments can be considered as a complex optimization problem since it needs to minimize flight time, energy consumption, and so on under many constrains. This paper proposes a two-stage multi-UAV path planning training framework based on MAPPO. In the first phase, the policy pretraining is conducted in a static environment to help agents learn fundamental navigation and obstacle avoidance skills. In the second phase, dynamic obstacles are included in the environment, and the pretrained model performs adaptive policy transfer. This scaffolding training paradigm effectively addresses the convergence challenges of direct training in dynamic environments, enhancing the robustness and generalization of agents in partially observable scenarios. To further improve learning efficiency of the agents, this paper introduces two key mechanisms. First, a dynamic reward weighting mechanism associated with environmental situation is designed, allowing autonomously adjust different proportions of the rewards. Second, a dynamic local perception module is incorporated into the agents' observation space, allowing the agents to focus on neighborhood observation. These improvements collectively form a dynamic coupling mechanism between the environment and agents, enhancing policy adaptability to complex scenarios. Simulation experiments demonstrate that: (1) Compared to the traditional MAPPO algorithm, the proposed method improves training convergence speed in dynamic environments. (2) In scenarios with randomly moving obstacles, the proposed method increases task completion rates in dynamic environments. (3) Ablation studies confirm that incorporating dynamic local observation space and dynamic weights improves task completion efficiency in dynamic scenarios.