Learning-Based AoI Minimization Through UAV-Assisted Data Distribution in Vehicular Networks

Uncrewed Aerial Vehicle (UAV) is extensively employed as a mobile base station in areas with inadequate cellular infrastructure to enhance the freshness of vehicle sensors. The Age of Information (AoI) is a metric utilized to characterize the freshness of information produced by vehicle sensors. This paper investigates the use of Uncrewed Aerial Vehicles (UAVs) as mobile base stations to enhance the freshness of vehicle sensor information in areas with inadequate cellular infrastructure. We focus on minimizing the Age of Information (AoI) and UAV energy consumption in a Vehicle-to-UAV (V2U) network within the Manhattan scenario. The challenge lies in jointly optimizing UAV trajectories and vehicle data packet scheduling amidst high vehicle mobility and limited communication range. To address this issue, we employ Reinforcement Learning (RL) to formulate the problem as a Markov Decision Process (MDP), proposing a Dueling Double Deep Q-Network (D3QN) method for trajectory and scheduling optimization. We also introduce Priority Experience Replay (PER) to improve reward acquisition for the UAV, addressing the issue of sparse rewards due to the expansive space for vehicle movement. Simulation results provide empirical evidence supporting the efficacy of the proposed algorithm in comparison to baseline policies.