Enhancing UAV Network Efficiency: A Multi-Agent DRL Approach for Joint User Association And Trajectory Optimization

In multi-UAV networks, user equipment (UE) needs to connect to UAVs for both uplink (UL) and ‎downlink (DL). Traditionally, UL and DL associations are coupled means a UE connects to the same ‎UAV for both. However, this approach is inefficient due to the mobility of UAVs and network heterogeneity. Full-duplex (FD) communication in UAV networks complicates this problem. This paper ‎introduces a novel decoupled UL-DL association (DUDe) framework. This allows each UE to associate ‎with different UAVs for UL and DL transmissions. Such decoupling improves flexibility and enhances ‎communication performance. However, UE association depends on UAV trajectories, making the prob-‎lem more complex. However, the dependence of UE association on UAV trajectories increases the ‎complexity. This work formulates a joint optimization problem to maximize the total sum-rate of UEs ‎in both UL and DL. To handle this uncertainty, a robust Partially Observable Markov Decision Process ‎‎(POMDP) model is used. This helps model the uncertain environment where UAVs do not have complete information about the system state. A Multi-Agent Deep Reinforcement Learning (MADRL) ap-‎approach is proposed to solve this problem. Each UAV selects its policy in a decentralized manner. The ‎training process is improved using a modified Proximal Policy Optimization (PPO) algorithm. It uses ‎deep reinforcement learning to efficiently manage UAV mobility and UE connectivity. The findings ‎suggest that DUDe-based associations outperform traditional coupled associations. It leads to better ‎spectral efficiency and higher network throughput. The proposed framework and algorithms are validated through simulations and real-world scenarios‎.