Optimizing Multi-Cell Selection Handover in Cellular Networks: A Deep Reinforcement Learning Approach

Handover (HO) is a critical component of mobility management in the 5th generation (5G) of communication networks, which ensures seamless connectivity and optimal communication performance for user equipment (UE) in motion across different cells. In previous studies, the deep reinforcement learning (DRL) techniques were employed to solve the HO problem. However, for most of these methods, the growing complexity in action space was not considered as the number of UEs increases, leading to inefficient model convergence and HO failures. To address this issue, this paper proposes a novel PPO-MH (Proximal Policy Optimization with Masking for Handover) model for multi-cell selection handover problem. This model calculates the action mask for each UE before each handover using the UE's measurement report, providing prior information for the decision-making process. By dynamically masking base stations (BSs) that do not meet the handover conditions, the model avoids invalid hand overs and improves sampling efficiency. Experimental results demonstrate that the PPO- MH outperforms the traditional PPO across various scenarios, ensuring Quality of Service (QoS) for UEs and reducing the handover frequency. Additionally, PPO- MH converges significantly faster than PPO, which validates the effectiveness of the action mask strategy. Benefiting from these advantages, our methods have broad potential applications, especially in scenarios requiring efficient resource management and low-latency handovers.