A Scale-Independent Multi Agent Deep Reinforcement Learning Framework for Wireless Resource Allocation in UAV-Based Logistics Networks

Unmanned aerial vehicles (UAVs) have demonstrated significant potential in improving the efficiency and accuracy of logistics distribution, particularly in challenging environments such as disaster zones and remote areas. Effective wireless resource allocation, including spectrum and power management, is crucial for enhancing the Quality-of-Service (QoS) of UAV-based logistics networks (UAV-LNs). However, devising an optimal resource allocation strategy for UAV-LNs presents challenges, especially when dealing with varying scales of UAV deployments. To address this challenge, this study proposes a scale-independent multi-agent deep reinforcement learning (MADRL) framework that can adapt to different DRL models. Initially, a decentralized Partially Observable Markov Decision Process (dec-POMDP) model is introduced to characterize UAV-LNs. Considering interactions and information exchange within its communication range, each agent's state incorporates local observations of both the environment and neighboring agents. Subsequently, to handle the variable dimensions of observed information, a state representation scheme is proposed to consolidate variable-dimensional information into a fixed-shape input variable, while ensuring the permutation of observed information remains inconsequential. Finally, the state representation scheme is integrated with three typical DRL methods to address the dec-POMDP problem. Simulation results show that the proposed framework is compatible with different method implementations, adapts to varying scales of UAV deployment, and consistently outperforms the baselines across diverse scenarios.