Long-Term Energy Efficiency Optimization in Wireless-Powered MEC Systems via Deep Reinforcement Learning

The rise of smart applications in wireless devices increasingly relies on mobile edge computing (MEC), where longterm system energy efficiency holds crucial significance for both green computing and application vendors. This paper focuses on long-term energy efficiency in a wireless power transferenabled MEC system. This system faces the challenges of timevarying channel states and stochastic task arrivals. We first formulate this problem to simultaneously optimize offloading, power transfer duration, and energy consumption, while ensuring device queue stability. We then introduce a novel algorithm based on Lyapunov-guided deep reinforcement learning, referred to as LyCNN-DRL. This approach efficiently handles the mixed integer non-linear programming problem by transforming it into a deterministic per-slot problem for online optimization, without needing prior knowledge of future conditions. Specifically, we tackle the problem by dividing it into resource allocation and binary offloading components, applying a convolutional neural network model for near-optimal offloading decisions, and obtaining the optimal solution for resource allocation. Simulation results show that LyCNN-DRL outperforms baseline algorithms, stabilizing MEC network task queues. Furthermore, we quantitatively derive the trade-off between energy efficiency and queue length, represented as <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$[O(1/V), O(V)]$</tex> with the variable <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$V$</tex>.