Deep Reinforcement Learning for Optimizing Multi-Hop Distributed Collaborative Task Offloading in R2X

Roadside units (RSUs) with strong sensing abilities enhance the viability of the RSU-to-Everything (R2X) paradigm, offering crucial infrastructure support for Mobile Edge Computing (MEC) that enables real-time data processing and reduced delay. Since RSUs collect a large volume of data but have limited computing capability, data analysis tasks are usually offloaded to other network nodes, such as the cloud, other RSUs, or even vehicles. The multi-hop distributed collaborative task offloading scheme is expected to achieve high resource utilization efficiency and low task delay in this scenario, despite increasing energy consumption in data transmission. However, the highly dynamic nature of the R2X network topology makes it challenging for a node to independently select the next hop and collaboratively allocate tasks to neighbors in a multi-hop transmission path. Specifically, offloading decisions made by an individual node are influenced not only by its immediate neighbors but also by other nodes along the multi-hop path, referred to in this paper as the effect value. Additionally, the heterogeneity in computing resources and link delays among network nodes further increases the difficulty. To address these challenges, we first apply a Long Short-Term Memory (LSTM) model to predict and update the neighbors for each node while considering effect values, allowing them to independently adapt to environmental changes. Then, we design a two-layer Deep Reinforcement Learning (DRL) algorithm for network nodes to make decisions. The first-layer DRL algorithm is implemented by RSUs to determine task offloading modes. When an RSU decides to offload tasks to multiple vehicles for collaborative computing, the second-layer DRL algorithm is used by a vehicle to select its next hop vehicle and allocate tasks. Simulation results show that our proposed approach effectively adapts to topology changes in complex and highly dynamic network environments. Compared with existing methods, this approach significantly improves task completion ratio and reduces processing delay.