AirFRL: Topology-Aware Decentralized Federated Reinforcement Learning for UAV Networks

Machine learning (ML) enhanced unmanned aerial vehicle (UAV) networks are envisioned to facilitate extensive applications in next-generation wireless networks. Due to the privacy concern and communication overhead in cloud-centric ML, federated reinforcement learning (FRL) enables UAVs to collaboratively train a policy model without disclosing raw observation data. However, the model aggregator in centralized FRL architecture poses various potential threats such as a single point of failure and is inappropriate to distributed networks with unreliable links and nodes. In this paper, we propose AirFRL, a topology-aware decentralized federated reinforcement learning framework for UAV-enabled networks. In AirFRL, we consider the topology dynamics influenced by nodes' mobility and communication quality and its impact on AirFRL. To accelerate training process and guarantee the model performance, we also incorporate the model compression to lighten the local model and introduce the consensus distance and data correlation to reflect the discrepancy between local models and local data. Furthermore, we propose an efficient algorithm to decide the optimal neighbour node selection and model compression ratio. A case study and numerical results demonstrate that AirFRL can achieve linear training speedup and guarantee the learning performance for UAV-enabled networks.