Deployment Design for Multi-UAV-Assisted IoT Networks: A Digital Twin-Driven Deep Reinforcement Learning Approach

In this paper, we address the multi-unmanned aerial vehicle (UAV)-assisted internet of things network (UAIoTN) in a partially-known 3-D urban environment. With randomly distributed and stationary ground nodes (GNs), the UAIoTN deployment is decoupled into two successive markov decision processes: the mission transfer stage (MTS) and the mission maintaining stage (MMS), forming a Bi-stage deployment (BiSD). We optimize UAV trajectories for data collection via a digital twin (DT)-empowered deep reinforcement learning approach. To achieve this, we propose a DT-driven double deep Q-learning network (DT-DDQN) and construct a novel DT-assisted training framework (DTTF) to enable model pre-training and updating. During mission execution, we enable obstacle avoidance and model the channel deterministically based on link blockage conditions. Furthermore, we streamline multi-UAV deployment through K-means-based mission division, assigning mission sub-regions and their GNs to UAVs, and integrating MTS and MMS for safe and efficient maneuvering and network coverage. Numerical results demonstrate that the proposed DT-DDQN-BiSD, operating in partially-known environments, outperforms baseline methods evaluated under ideal fully-known conditions, in terms of mission execution time, DRL model convergence, and generalization.