Joint Trajectory and Spectrum Optimization in Advanced Air Mobility via Multi-Agent Deep Reinforcement Learning

Urban air transportation is undergoing a revolutionary transformation through the integration of Urban Air Mobility (UAM) and Advanced Air Mobility (AAM) systems. One big challenge is that new aerial vehicles (AV) will quickly saturate the already crowded aviation spectrum, which is an essential resource to ensure reliable communications for safe air operations. In this paper, we consider an air transportation system where multiple AV s are operated to transport passengers or cargo from different sources to destinations. During the flight, the minimum communication Quality of Service (QoS) must be achieved at all times to ensure flight safety. Our objective is to minimize the total mission completion time by jointly optimizing the trajectories, velocities, and spectrum allocation for all AV s. We formulate the optimization problem as a multi-stage Markov game where the optimization variables are coupled together. A multi-agent deep Reinforcement learning VD3QN algorithm is proposed to enable cooperative learning among AV s for both trajectory planning and resource optimization. Additionally, we propose an orthogonal multiple access with Space-Time A * algorithm as a non-learning-based solution. Extensive simulation results show that our learning-based solution outperforms both the non-learning-based solution and other learning-based approaches like Qmix under different parameters.