A Policy Reuse Reinforcement Learning Framework for Hard Latency Constrained Resource Scheduling

In the forthcoming 6G era, extend reality (XR) has been regarded as an emerging application for ultra-reliable and low latency communications (URLLC) with new traffic characteristics and more stringent requirements. In addition to the quasi-periodical traffic in XR, burst traffic with both large frame size and random arrivals in some real world low latency communication scenarios has become the leading cause of network congestion or even collapse, and there still lacks an efficient algorithm for the resource scheduling problem under burst traffic with hard latency constraints. We propose a policy reuse reinforcement learning framework for resource scheduling with hard latency constraints (PRRL-RSHLC), which maximizes the hard-latency constrained effective throughout (HLC-ET) of users. The proposed algorithm reuses polices from both old policies learned under other similar environments and domain-knowledge-based (DK) policies constructed using expert knowledge to improve the performance. Simulations show that PRRL-RSHLC can achieve superior performance with faster convergence speed compared to baseline aIgorithms.