Online Service Selection and Task Offloading in UAV-Assisted MEC via Deep Reinforcement Learning

Unmanned Aerial Vehicle (UAV)-assisted Mobile Edge Computing (MEC) combines UAV technology and MEC paradigms. However, it is challenging to guarantee the User Terminals’ (UTs) Quality of Experience (QoE) simply by offloading tasks to the UAV for assisted computation. This paper studies a UAV-assisted heterogeneous edge computing network, where a cache-enabled UAV dynamically responds to online service requests from MEC servers. The QoE is defined as a weighted sum of latency reduction and energy savings at UTs. To maximize UAV energy efficiency, a joint optimization problem is formulated, involving online service selection, task offloading, CPU frequency allocation, UAV trajectory, and hovering scheduling. The problem is modeled as a multi-stage stochastic mixed-integer nonlinear program. A hierarchical framework combining Deep Reinforcement Learning (DRL) and Lyapunov optimization is proposed. The Lyapunov method analyzes task queues and balances queue lengths with utility, while Deep Deterministic Policy Gradient (DDPG) derives online services decision policies. UAV trajectory and hovering are further optimized based on the decisions. Simulation results show that the proposed method significantly outperforms benchmark schemes in terms of UAV energy efficiency.