Distributed UAVs Deployment for Large-Scale Traffic Based on Federated Reinforcement Learning

In collaborative Unmanned Aerial Vehicles (UAVs) swarm perception tasks, centralized deployment algorithms are only applicable to limited numbers of UAVs since they are susceptible to changes in the state-action space dimension. However, the increase of task demands leads to an explosion in the state-action space of centralized algorithms, which results in convergence difficulties. To reduce the training complexity of UAV swarm deployment and improve the system robustness, this paper proposes a Fusion-Sharing based Distributed UAVs Deployment (FSDUD) algorithm. First, we employ the mean-field theory to simplify the state interaction of local UAVs and upload their policy network parameters to the cloud server. Then, an experience fusion mechanism based on the fusion of action value labels is proposed for aggregating deployment strategies in other localized areas. Finally, the experience sharing mechanism is used to adjust the feature size and ensure consistency between the input layer and the initial feature vector. The priori knowledge is obtained by fusion and then shared asynchronously with other local networks to improve the training process and the robustness of the system. Simulation results show that the proposed FSDUD algorithm can effectively reduce the training time of UAVs deployment by 57% while improving robustness of the system for large-scale urban traffic.