Distributed Computation Offloading in Vehicular Edge Networks Using Proactive Service Reservations and Multi-Agent Learning

Future <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">vehicular edge computing</i> (VEC) networks are projected to generate massive amounts of computation tasks which exhibit dynamic spatio-temporal heterogeneity. Vehicular users will greatly benefit if the VEC network provides the following information a priori: a) estimates of the delays along the paths and b) amounts of network resources that will be assigned. Existing works did not provide such prior information and only dealt with real-time computation offloading. In this paper, we propose a novel distributed computation offloading strategy in which the vehicular users independently make proactive service reservations across the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">road side units</i> (RSUs) along their paths. Such reservations are based on the computation offloading delays reported by the RSUs. These delays depend on the optimal network resources assigned by the RSUs to the vehicular users over different reservation slots. In order to determine the optimal assignment of network resources, we propose a new <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">multi-agent deep reinforcement learning</i> (MADRL) problem where the RSUs act as agents. The objective of the MADRL problem is to maximize the mean number of offloaded tasks. This objective is found as the outcome of the proactive service reservation problem solved by each vehicular user when it is injected into the VEC network. The MADRL problem is approximated using a centralized-critic-distributed-actors model. The proactive service reservation introduces time coupling in the calculation of the rewards, therefore a novel training algorithm is proposed for solving the MADRL problem. Compared to some benchmark real-time computation offloading schemes, the proposed work benefited the vehicular users significantly by providing prior estimates of the delays along their paths. Specifically, the percentage of offloaded tasks achieving the highest utility was <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$14-20\,\%$</tex-math></inline-formula> higher, b) the computation offloading delays experienced by the vehicular users along the path were the lowest, and c) the utilization of network resources was lowest compared to the benchmark schemes.