Age of Information and Energy-Efficiency Optimization in RIS-Assisted Vehicular Edge Computing via Hierarchical Deep Reinforcement Learning

With the rapid development of the Internet of Vehicles (IoV) and the increasing demand for real-time data processing, efficient resource allocation (RA) has become a critical challenge in vehicular edge computing (VEC) systems. In this article, we present a novel hybrid deep reinforcement learning (DRL) framework for optimizing RA in reconfigurable intelligent surface (RIS)-assisted VEC systems. The proposed framework addresses the dual challenges of minimizing the Age of Information (AoI) and maximizing energy efficiency in dynamic vehicular networks. By leveraging a hierarchical decomposition strategy, the framework separates the optimization of RIS phase shifts and federated power allocation, enabling efficient management of high-dimensional decision spaces. The upper layer employs a twin delayed deep deterministic policy gradient (TD3)-based RIS controller to optimize phase shifts, while the lower layer utilizes a federated multiagent deep deterministic policy gradient (FMADDPG) algorithm for power allocation. To enhance policy awareness and accelerate convergence, a trajectory pool of upper-layer RIS phase shifts is constructed and embedded into the lower-layer state space, thereby achieving soft-coupled collaboration among hierarchical policies. Simulation results under realistic urban mobility patterns and 3-D geometric channel models demonstrate that the proposed framework significantly outperforms benchmark algorithms, including soft actor–critic (SAC), QMIX, and block coordinate descent (BCD), in terms of AoI reduction, energy efficiency, and communication throughput.