A Bi-Level Scheme for Mixed-Motive and Energy-Efficient Task Offloading in Vehicular Edge Computing Systems

Edge computing is considered as a promising paradigm to support vehicular applications in the upcoming sixth-generation (6G) vehicular networks. In the context of vehicular edge computing (VEC), the self-interested vehicular users and edge servers work towards incongruous goals. Such mixed-motive setting is detrimental to the collective good, sometimes leading to social dilemmas. To resolve such a conflict, we first formulate a bi-level optimization problem to model mixed-motive task offloading. In this case, vehicular users aim to improve energy efficiency under strict low-latency requirements, whereas edge servers attempt to increase serving efficiency. To address it, we propose a scheme based on bi-level reinforcement learning, i.e., bi-level multi-agent actor-critic (BLMAAC) framework. Specifically, upper-level edge servers make iterative optimization under the best responses of lower-level vehicular users, which can be regarded as a Stackelberg game. Theoretically, we identify the conditions and prove the convergence of the framework that is able to reach Stackelberg equilibrium strategy. By numerical evaluation, the high-utilization edge servers and energy-efficient vehicular users demonstrate the superiority of the bi-level structure. Moreover, the proposed scheme outperforms other actor-critic based learning algorithms and two-stage methods exploring Nash equilibrium strategy.