Efficient user scheduling in mmWave networks: leveraging knowledge transfer with channel knowledge map

This paper proposes a Deep Reinforcement Learning (DRL) algorithm for user scheduling in Millimeter Wave (mmWave) networks, which utilizes Channel Knowledge Map (CKM) for knowledge transfer to enhance the learning of scheduling strategies. The user scheduling and link configuration problems are modeled as a multi-queue system. Each queue represents the data demand of an individual user. This setup allows the base station to make dynamic scheduling decisions based on changing environmental conditions. This approach facilitates efficient management of user-specific requirements while addressing the challenges posed by dynamic network environments. Our model incorporates relay selection, codebook selection, and beam tracking to support flexible and efficient resource allocation. In contrast to traditional channel model-based optimisation, we design algorithms for scheduling policy pre-training using CKMs, which provide information about the channel between specific pairs of locations. Specifically, we assume that the CKM is fully available to allow the complex scheduling network to have a better starting point or follow a more favourable gradient direction through knowledge migration. This integration of CKM with knowledge transfer significantly accelerates DRL convergence and enhances performance stability. Simulation results confirmed the effectiveness of the proposed approach. Relative to the baseline methods, integrating CKM with knowledge transfer accelerated the convergence of the DRL algorithm by approximately 20%, maintained the delay within 30 milliseconds, and reduced the average queue length by nearly 30%.