Multiagent Deep Reinforcement Learning-Based Hierarchical Scheduling in Heterogeneous UAV-Enabled Vehicular Networks

Current vehicular network systems often encounter significant challenges in dynamic task offloading and communication resource optimization, particularly in dynamic environments with varying computational demands. To address the challenges, we propose a novel two-tier Unmanned Aerial Vehicle (UAV)-assisted architecture architecture where observation rotary-wing UAVs are stationed above roadways to collect and transmit vehicle task information to relay rotary-wing UAVs hovering over urban areas, which further offload the tasks to remote mobile edge computing (MEC) servers for execution and return the processed results. This approach enhances data transmission rates and optimizes network coverage, enabling seamless computational task offloading from vehicles to edge while reducing latency. To solve the multi-layer vehicle task scheduling problem within heterogeneous intelligent UAV architectures, we first formulate an optimization problem aimed at minimizing latency and maximizing load balancing, which is further analyzed by a Markov decision process (MDP). Then, We propose a multi-agent deep reinforcement learning-based hierarchical scheduling algorithm in heterogeneous UAVs enabled vehicular network (MAHHV) with the centralized training and decentralized execution (CTDE) framework to pursue the optimal hierarchical offloading policy. Additionally, we integrate a Gated Recurrent Unit (GRU) network to enhance the temporal dependency modeling and improve decision-making efficiency in dynamic environments. Numerical results demonstrate that the proposed algorithm outperforms the best baseline multi-agent proximal policy optimization (MAPPO) by 0.6% on average across varying numbers of vehicles, validating its substantial potential for advancing intelligent vehicular systems and enhancing the efficacy of distributed computing in real-time applications.