DT-LNS: Digital-Twin-Based Low-Risk Network Slicing Using Safe Reinforcement Learning

Network slicing (NS) is a key technology to cost-effectively meet diverse service level agreement (SLA) demands of the Internet of Everything communication. Thanks to high-fidelity network modeling capabilities and flexible feedback optimization techniques, digital twins (DTs) and reinforcement learning (RL) have been applied to dynamic NS management. However, most existing DTs lack the ability of predictive uncertainty evaluations, and tend to be overconfident on the unknown network environment. For classical RL, it is exceedingly intractable to maintain high-stable NS performances in dynamic networks. To address those problems, we propose a DT-based low-risk NS (DT-LNS) framework and method using the safe RL. In the safe RL, a DT using deep neural networks with the data-model uncertainty analysis is adopted to predict NS performances and provide predictive uncertainties. Further, the RL is used to select low-risk NS configuration actions by preverifying the SLA violation risk of candidate actions from the RL and the reference action subspace via DTs. The proposed DT-LNS method can keep the high-SLA satisfaction rate (SSR), reduce the performance jitters, and improve the convergence speed. Compared with the six classic NS configuration methods, including round robin, deep Q network, advantage actor-critic, deep deterministic policy gradient, and advanced RL, assisted with the DT-based model pretraining and the state prediction, the average percentage gain of the proposed method is 7.84%, 93.58%, 65.63%, 84.20%, and 90.27%, regarding the performances of the average SSR, SSR jitter, delay jitter, data rate jitter, and the convergence speed, respectively.