A Task-Centered Algorithm for AAV-Assisted Communications Based on Deep Reinforcement Learning

Autonomous aerial vehicles (AAVs) can be harnessed to provide temporary communication resources and act as a relay to assist users in transmitting their tasks to an external base station or cloud. How to allocate AAV resources to assist users has become an important topic in AAV-assisted communications. Although existing studies have considered terrain characteristics, communication throughput and user equality, task properties have often been overlooked, which would impact the overall quality of service. Due to the multiple complex attributes of tasks and dynamic complexity of the environment, optimization often involves high dimensionality and dynamism, making it difficult for traditional algorithms to distinguish primary tasks and effectively transmit important information. We thus propose a multiconstrained nonconvex joint optimization problem modeled as a partially observed Markov decision process, which brings the resource optimization challenges of AAV communication resources. This article further proposes a deep reinforcement learning (DRL)-based algorithm named weighted-K-means-DDPG (WK-Means-DDPG). The innovation of this algorithm lies in the combination of the traditional Weighted K-Means algorithm and DRL algorithm deep deterministic policy gradient (DDPG) to solve the deployment and resource allocation problems of AAVs and improve the performance of the reinforcement learning algorithm. Simulation results show that our algorithm outperforms state of the art and baseline algorithms in transmitting important information and can be generalized to different size of areas, number of users, and number of AAVs.