Multi-UAV Collaborative Path Planning Based on CycA-MASAC Reinforcement Learning in GPS-Denied Environment

This paper addresses collaborative path planning for UAVs in GPS-denied environments by proposing an improved multi-agent deep reinforcement learning algorithm, Cycloidal Annealing - Multi-Agent Soft Actor-Critic (CycA-MASAC). By designing a reward function for UAV collaborative flight and a Cycloidal Annealing learning-rate algorithm, incorporating Partially Observable Markov Decision Process (POMDP) theory and UAV dynamics equations, a multi-UAV path-planning scenario with obstacle avoidance in airspace was constructed. Performance metrics, including task completion rate, formation retention rate, flight time, flight distance, and energy consumption, were designed to comprehensively assess the algorithm’s performance. Comparative tests on reward functions, sensitivity tests on different formation modes, and collaborative strategy tests for UAVs were conducted. Experimental results show that the CycA-MASAC reinforcement learning method outperforms the traditional MASAC algorithm in terms of faster convergence, stronger stability, and a 10.01% increase in task completion rate and a 17.17% improvement in formation retention rate compared to the original algorithm. In addition, flight strategy testing has shown that the CycA-MASAC algorithm proposed in this paper effectively balances flight costs and safety, demonstrating excellent performance in both swarm coordination and flight safety.