Age of Information Minimization in Vehicular Edge Computing Networks: A Mask-Assisted Hybrid PPO-Based Method

With the widespread deployment of various emerging intelligent applications, information timeliness is crucial for intelligent decision-making in vehicular networks, where vehicular edge computing (VEC) has become an important paradigm to enhance computing capabilities by offloading tasks to edge nodes. To promote the information timeliness in VEC, an optimization problem is formulated to minimize the age of information (AoI) by jointly optimizing task offloading and subcarrier allocation. Due to the time-varying channel and the coupling of the continuous and discrete optimization variables, the problem exhibits non-convexity, which is difficult to solve using traditional mathematical optimization methods. To efficiently tackle this challenge, we employ a hybrid proximal policy optimization (HPPO)-based deep reinforcement learning (DRL) method by designing the mixed action space involving both continuous and discrete variables. Moreover, an action masking mechanism is designed to filter out invalid actions in the action space caused by limitations in the effective communication distance between vehicles. As a result, a mask-assisted HPPO (MHPPO) method is proposed by integrating the action masking mechanism into the HPPO. Simulation results show that the proposed MHPPO method achieves an approximately 28.9% reduction in AoI compared with the HPPO method and about a 23% reduction compared with the mask-assisted deep deterministic policy gradient (MDDPG).