When Learning Meets Dynamics: Distributed User Connectivity Maximization in AAV-Based Communication Networks

Distributed network management over Autonomous Aerial Vehicle (AAV) based communication networks (ACNs) has attracted increasing research attention. In this work, we study a distributed user connectivity maximization problem in a ACN. The work features a horizontal study over different levels of information exchange during the distributed iteration and a consideration of dynamic AAV set and user distributions, which are not well addressed in existing work. Specifically, the studied problem is first formulated into a time-coupled mixed-integer non-convex optimization problem. The problem aims to maximize the accumulated number of served users in a slotted time horizon by optimizing AAV trajectories and determining user association. A heuristic two-stage AAV-user association policy is proposed to faster determine the user connectivity. To tackle the NP-hard problem in a scalable manner, the distributed user connectivity maximization <xref ref-type="algorithm" rid="alg1" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">algorithm 1</xref> (DUCM-1) is proposed under a multi-agent deep Q learning (MA-DQL) framework. DUCM-1 emphasizes on the design of different information exchange levels and evaluating how they impact the learning convergence with stationary and dynamic user distribution. To comply with the AAV dynamics, DUCM-2 algorithm is devoted to autonomously handling arbitrary quit’s and join-in’s of AAVs in a considered time horizon. Extensive simulations are conducted <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i)</i> to conclude that exchanging state information with a deliberated task-specific reward function design yields the best convergence performance, and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">ii)</i> to show the efficacy and robustness of DUCM-2 against the dynamics.