Bi-Level Reinforcement Learning-Based Task Offloading for Delay Optimization in Wireless Powered Mobile Edge Computing

Wireless powered mobile edge computing combines the benefits of wireless power transfer and mobile edge computing, enabling mobile devices (MDs) to overcome energy and computational constraints by obtaining energy from hybrid access point (HAP) and offloading computation tasks to HAP. However, given the devices' half-duplex nature, conflicts arise between time allocation and task offloading. Current research lacks a comprehensive examination of the joint impact of task offloading and time allocation on long-term delay. Furthermore, most existing centralized solutions may encounter issues such as dimensionality explosion and so on. This paper thoroughly investigates the task offloading problem in wireless powered mobile edge computing systems, focusing on minimizing long-term task completion delay under delay and energy constraints. To address this problem, we reformulate the task offloading process as a bi-level Markov decision process and convert it into an optimization problem of finding the optimal policy. Subsequently, we introduce a Bi-level Actor-Critic-based Task Offloading Approach (BiAC-TOA), where MDs and HAP leverage centrally trained agents for decisions on task offloading and time allocation in a decentralized manner. Finally, a series of experiments confirm the superiority of our proposed approach.