Max-Min Fairness-Oriented Navigation and Scheduling for UAV-Enabled Wireless Networks via Deep Reinforcement Learning

As modern wireless systems transit towards the sixth-generation (6G), uncrewed aerial vehicles (UAVs) are becoming essential in realizing ubiquitous coverage due to their fully controllable mobility and high transmission efficiency. To achieve max-min fairness among users, we aim to maximize the minimum throughput by optimizing the UAV trajectory and transmission scheduling in real-world scenarios where the UAV has access only to <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">causal</i> channel state information (CSI). However, max-min fairness is involved with the user-to-user dependency across the time horizon, which makes it challenging to maximize the minimum throughput without prior knowledge of the future CSI. To tackle this issue, we propose a bisection-integrated multi-step dueling double deep Q-network (Bi-D3QN) algorithm by combining a bisection method with a multi-step D3QN framework. The Bi-D3QN algorithm can solve the minimum throughput maximization problem by alternating between (1) finding optimal policies that minimize the UAV’s mission completion time for a given minimum throughput requirement through the multi-step D3QN and (2) evaluating the feasibility of the trained policy, followed by updating the minimum throughput requirement through the bisection method. Simulation results demonstrate that the proposed Bi-D3QN algorithm achieves a higher minimum throughput compared to existing benchmark schemes through real-time interaction with the environment.