Holistic Mobility Management leveraging Risk Averse Reinforcement Learning

The trend towards denser base station deployment and multi-band operations in emerging cellular networks has made mobility management and handover (HO) optimization a formidable challenge. The challenge is further aggravated by the scarcity of practical multi-objective mobility management solutions optimizing both intra and inter frequency HO. This paper presents a holistic multi-objective mobility management solution for both intra and inter frequency HO employing multiple parameters of standardized HO events A2, A3, and A5. We formulate a multi-objective optimization problem to determine the optimal parameter settings that jointly optimize four key performance indicators: number of HO failures, HO latency, signaling overhead and number of radio link failures. We leverage soft actor-critic reinforcement learning (RL) to solve the multi-objective problem. To mitigate the risk of performance deterioration resulting from direct interactions between live network and RL-agent during training, this paper proposes a mobility management framework that develops and employs a digital twin (DT) as the training environment. To develop a cellular network DT for mobility management and HO optimization, we present a tri-pronged approach including realistic network deployment, realistic user mobility and 3GPP HO events. Results show that the proposed DT-trained RL solution for the multi-objective optimization can converge 7x faster than the brute force method with negligible loss in the value of the objective function. An analysis of the individual KPI values reveal a strong trade-off between HO signaling overhead and radio link failures.