Multi-Agent Deep Reinforcement Learning-Based UAV Trajectory Optimization for Data Collection

Deploying low-cost sensors to collect large-scale regional data is necessary in numerous applications such as environmental monitoring. However, the manual collection of sensor data is cost-ineffective. With the development of the Internet of Things (IoT) and unmanned aerial vehicle (UAV) technologies, the collection of sensor data by UAVs has emerged as a promising approach. Therefore, it is often crucial to optimally deploy UAVs for efficient data collection to ensure low Age of Information (AoI) and low energy consumption. To describe data freshness and account for volume, we propose a new data value measurement metric termed \({\rm{eAoI}}\), inspired by AoI. To consider both the data value and energy consumption, we further define a new concept: data value per energy unit. We consider the scenario where distributed sensors generate data continuously, while a set of UAVs are deployed for efficient data collection. We aim to maximize data collection efficiency, namely, the overall eAoI gain per energy unit cost throughout the collection operation lifecycle. We propose K-MADDPG, where all actors share a single Critic, enhanced with Kolmogorov–Arnold networks (KAN). The UAVs in the K-MADDPG framework had distributed decision-making capability, and could dynamically adjust their trajectory strategy according to the environment. Simulation results demonstrate that K-MADDPG exhibits significant improvement in terms of both convergence speed and performance.