QRMP-DQN Empowered Task Offloading and Resource Allocation for the STAR-RIS Assisted MEC Systems

This paper presents a novel simultaneously transmitting and reflecting reconfigurable intelligent surfaces (STAR-RIS) assisted mobile edge computing (MEC) system. We employ the semi-grant-free (SGF) non-orthogonal multiple access (NOMA) to improve the system's spectrum and energy efficiency, and the STAR-RIS to enhance the uplink communication from mobile users to the base station (BS). The joint task offloading and resource allocation (JTORA) for the STAR-RIS-assisted SGF-NOMA MEC system with imperfect channel state information is investigated to minimize the average energy consumption. Considering user mobility and dynamic arrival tasks, a JTORA framework comprised of reinforcement learning and a convex optimization module is proposed to tackle this resultant optimization problem. Specifically, a novel quantile regression multi-pass deep Q-network (QRMP-DQN) algorithm is proposed to deal with the hybrid discrete-continuous action structure of MUs and STAR-RIS. Moreover, the convex optimization module adopts the Karush-Kuhn Tucker conditions to derive the optimal computing resource allocation scheme. Simulation results unveil that: 1) the proposed framework can effectively solve the dynamic optimization problem and outperform the conventional DQN algorithm; 2) the STAR-RIS can significantly improve the performance of the SGF-NOMA MEC system compared to the benchmark cases.