Spectrum Resource Allocation for Large-Scale UAV Clusters in Complex Electromagnetic Environments: An Improved Deep Reinforcement Learning Method

This paper proposes a spectrum resource allocation method based on distributional reinforcement learning (C51-SRAA) to optimize spectrum resource allocation for large-scale UAV swarms. It achieves joint allocation in both frequency and power domains to maximize spectrum resource utilization. Firstly, the method addresses dynamic communication traffic changes between ground stations and UAV clusters through an adaptive spectrum allocation scheme, enabling dynamic spectrum access and transmission power adjustment, significantly enhancing communication efficiency. Secondly, by representing the value of each action as a probability distribution rather than a single value, the method captures more comprehensive policy information, improving adaptability to environmental changes. Additionally, the method employs a loss function based on Kullback-Leibler divergence to enhance the learning capability of Q-value distributions, overcoming the limitations of traditional mean squared error (MSE) in handling probability distributions. The research results demonstrate the algorithm's superiority in spectrum resource utilization and communication delay.