Average System Cost Minimization-Based Joint UAV Deployment and Resource Allocation

Unmanned aerial vehicles (UAVs) are expected to act as aerial relays which forwards data packets for ground users (GUs) leveraging their advantages of low cost, high flexibility and maneuverability. One challenging problem in UAV-assisted cellular systems is how to design the efficient UAV deployment, GU association and resource allocation strategy which achieves system performance optimization. In this paper, we address the data transmission problem in a UAV-assisted cellular system with the knowledge of statistical GU positions. Stressing the energy consumption of base station (BS) and UAVs, and the cost of UAVs, we formulate the joint UAV deployment, GU association and power allocation problem as a constrained system cost minimization problem. To solve the formulated problem, we decouple it into three subproblems, i.e., UAV deployment, GU association and power allocation subproblem. Then, the UAV deployment subproblem is modeled as a Markov decision process (MDP), and an embedded multi-agent double deep $\mathbf{Q}$ network (DDQN) algorithm is proposed. Specifically, given the state and action of the MDP, we formulate and solve the power allocation subproblem and determine the transmit power of the UAVs by applying the Lagrange dual method-based algorithm. The GU association subproblem is then tackled by utilizing a proposed Kuhn-Munkres (K-M) algorithm-based scheme. Based on the obtained power allocation and GU association strategy, the reward of the MDP can be computed and the UAV deployment strategy is determined which maximizes the long-term average reward. Simulation results demonstrate the effectiveness of the proposed algorithms.