Deep Deterministic Policy Gradient for Handovers in Mobile Networks with Transparent UAV Relays

In this paper, we introduce a novel framework jointly managing handovers of user equipments (UEs) and Unmanned Aerial Vehicles (UAVs) serving the UEs. The goal is to maximize the sum capacity of the UEs while considering a cost related to the handovers. To this end, we introduce a novel approach based on deep deterministic policy gradient (DDPG) adjusting the Cell Individual Offset (CIO) for handovers of the UEs among the UAVs and ground base stations (GBSs) as well as handovers of the UAVs among the GBSs. The UAVs playing the role of relays often face challenges related to the implementation cost and energy limitations. To address these challenges, the UAVs should operate in a transparent relaying mode. In such mode, unfortunately, the channels between the UEs and the UAVs are unknown as the transparent relays lack any communication control-related functionalities. Therefore, we adopt a deep neural network (DNN) to predict the channel qualities among the UEs and the UAVs for the handover purposes. We demonstrate that the proposal significantly increases the sum capacity of the UEs by dozens of percent and even reduces the number of handovers compared to state-of-the-art works. At the same time, the proposed DDPG-based CIO setting reduces a gap in the sum capacity between the predicted and the optimal (but practically not feasible) case with perfectly known channels among UEs and UAVs. Hence, the proposal is suitable for practical scenarios with not perfectly accurate channel quality information.