Federated Twin Delayed Deep Deterministic Policy Gradient for Delay and Energy Consumption Optimization in Urban Air Mobility with UAV-Assisted MEC

With the rapid expansion of urban populations and the accelerated pace of urbanization, the concept of urban air mobility (UAM) has emerged. During flights, UAM aircraft need to transmit real-time sensing information to base stations for further processing and analysis. Large-scale real-time data require leveraging the computing capabilities of edge servers at the network edge to reduce transmission delay and energy consumption of UAM aircraft. In cases where edge servers are unable to process information, an unmanned aerial vehicle (UAV) equipped with computing capabilities and operating in low-altitude airspace can serve as a relay to assist in communication and computation. Due to the limited payloads and flight times of UAVs and UAM aircraft, delay and energy consumption within the system pose significant challenges. To tackle these challenges, two fundamental objectives have been proposed: minimizing delay and minimizing energy consumption. Furthermore, an optimization problem has been proposed to minimize the weighted sum of delay and energy consumption. Then, a UAM federated twin delayed deep deterministic policy gradient (UF-TD3) algorithm has been proposed to solve the original problems characterized by complex, non-convex, and inseparable variables. Simulation results show that the proposed UF-TD3 algorithm converges quickly and significantly outperforms four other baseline algorithms in optimizing delay and energy consumption performance. Moreover, compared to the conventional delay minimization strategy and energy minimization strategy, the proposed strategy of minimizing the weighted sum of delay and energy consumption can reduce the delay by 63.8% and reduce energy by 73.96%.