Personalized Federated Reinforcement Learning for Multi-AAV Assisted Edge Computing

In this paper, we propose an innovative autonomous aerial vehicle (AAV)-assisted computation offloading scheme, where multiple AAVs with heterogeneous computational capabilities collaborate to provide task offloading services for mobile users (MUs). Our objective is to minimize the weighted sum of the MUs’ maximum task processing delay and the AAVs’ average energy consumption by jointly optimizing the AAVs’ service locations, velocities, user association and offloading policies. We model this problem as a fully decentralized partially observable Markov decision process (Dec-POMDP) and propose a personalized federated reinforcement learning (FRL) algorithm, named federated knowledge distillation based parameterized deep Q-network (FKD-PDQN), to collaboratively optimize the AAVs’ policies in a decentralized and cost-efficient manner. In this approach, each AAV optimizes its local policy using PDQN, and updates its model by aggregating the distilled knowledge from the neighboring AAVs through an attention mechanism. Simulation results reveal that the proposed algorithm effectively reduces the average delay, energy consumption, and communication overhead, offering significant improvements over baseline algorithms such as Independent PDQN (IPDQN), Federated PDQN (FPDQN), and Krum.