Multi-Level Deep Reinforcement Learning-based Edge Caching Strategies in Vehicular Networks

Recent advances in internet of vehicles (IoV) have highlighted the importance of vehicular networks in intelligent transportation systems (ITS). The rapid growth in the number of connected sensors and users requires a better networking scheme. However, the dynamic nature of actual networks poses significant challenges. In addition, content delivery should occur within a specific time frame, known as the delivery deadline, and conventional quality of service (QoS) metrics must be met. In this paper, we implemented cooperative edge caching strategies aimed at reducing the weighted average of energy consumption and content access delay in a vehicular networks while considering the data delivery deadline. By utilizing an abstract layer, along with federated deep reinforcement learning (DRL) and hierarchical DRL, we enhanced the efficiency of the caching scheme. This approach mitigates the challenges posed by the non-stationary nature of file popularity distribution and user behaviors. Our results show that the use of DRL agents in federated and hierarchical structures allows us to develop an effective caching strategy without requiring prior knowledge, such as file popularity distribution or user request patterns while improving cache hit rates, reducing data transmission power and access delay in IoV networks.