Joint Trajectory Design and Resource Allocation for Energy-Efficient Multi-UAV Assisted Vehicular Networks: An IKPP Approach

This paper focuses on the energy-efficient unmanned aerial vehicles (UAVs) assisted vehicular networks, where multiple rotary-wing UAVs are deployed to provide uplink service to numerous ground vehicles. The objective is to maximize the long-term system energy efficiency (EE) over the entire service period, through jointly optimizing the vehicle-UAV association, the sub-carrier assignment, the power control of vehicles and the trajectory design of UAVs. The formulated problem involves the limitations on the transmit power of vehicles and the propulsion power of UAVs, the quality of service (QoS) requirements of vehicles, the UAV movement constraints, and co-channel interference among vehicles. This makes the problem a mixed integer non-convex fractional programming problem accompanied by a mass of variables and diverse constraints, which is difficult to be solved within the polynomial time through traditional optimization methods. To cope with the timely decision-making requirement and dynamic moving scenario, we opt to the deep reinforcement learning (DRL) approach. To do so, the formulated problem is first transformed to a Markov decision process (MDP). Then, an improved k-means proximal policy optimization (IKPP) algorithm is proposed to solve the MDP problem. The proposed algorithm involves action reconstruction, the improved k-means algorithm, and proximal policy optimization-clip algorithm, which can help obtain the solutions with low complexity. Simulation results demonstrate the convergence, scalability and real-time of the proposed algorithm, along with its performance advantage over other benchmark algorithms.