Hierarchical Intelligence Enabled Joint RAN Slicing and MAC Scheduling for SLA Guarantee

As a key technology in beyond 5G and future 6G communications, RAN slicing can realize differentiated service level agreement (SLA) guarantees. In this paper, we investigate the multi-slice multi-user RAN slicing. A dynamic bandwidth allocation scheme for RAN slicing is proposed based on hierarchical intelligence, where bandwidth pre-allocation and hyper-parameter tuning for MAC layer schedulers are jointly optimized to maximize the system utility, i.e., the weighted sum of spectrum efficiency (SE) and SLA satisfaction ratio (SSR) of different slices. The problem is formulated as a twin-time scale Markov decision process (MDP), where the bandwidth pre-allocation and the scheduler parameter tuning are performed on a long-term scale (e.g., seconds) and a short-term scale (e.g., 100 ms), respectively, for which we propose a hierarchical twin-time scale Dueling deep Q learning network (TTS-DDQN) algorithm. A new reward-clipping mechanism is proposed to get a better trade-off between stabilized training and higher system utility. In order to improve the robustness to time-varying traffic patterns and non-stationary dynamic environments, we further propose a traffic-aware module for more efficient sampling of the experience pool, and a variational adversarial inverse reinforcement learning (VAIRL) module for reward automation design. Extensive simulations show that the traffic-aware TTS-DDQN in stationary scenarios and the VAIRL module embedded TTS-DDQN in non-stationary scenarios outperform existing typical DQN-based algorithms, hard slicing and non-slicing, etc.