Energy-Aware Multi-Agent K-hop Proximal Policy Optimization for Mission-Oriented Drone Networks

Mission-oriented drone networks are widely used in tasks challenging for humans to conduct and for those benefiting from drones’ easy deployment and mobility. Drone limitations, such as battery life and the communication bottleneck between the drones and the base station, significantly impact network performance and mission success. Mission execution strategy design for these dynamic environments is complex, requiring efficient drone collaboration while considering their limitations. While multi-agent reinforcement learning has been used to address these issues, to the best of our knowledge, our work is the first to use the Proximal Policy Optimization algorithm with localized information to tackle these challenges. Our proposed model, Energy-Aware Multi-Agent K-hop Proximal Policy Optimization (EMA-KPPO), uses K-hop localized information to alleviate the communication bottleneck and account for limited drone battery life. Simulation results indicate that the 1-hop version performs comparably to the 2-hop version across various network sizes, validating the effectiveness of using localized information. The proposed EMA-KPPO consistently achieved over 95% success rates at task densities of 30% and above, with stable trends in success rates and mean completion steps across different grid sizes.