Research on Multi-Timescale Scheduling in Cloudnative with Deep Reinforcement Learning

Cloud-native development is flexible and scalable, but resource scheduling still faces the challenge of improving resource utilization in the long and short term. traditional methods are based on static rule-based scheduling, which is not intelligent enough, leading to unbalanced resource utilization and affecting the system QoS. in this paper, we propose a multi-timescale resource balanced scheduling method based on intelligent decision making to coordinate resource scheduling requirements on different timescales of the long term, medium term and short term. By formalizing the scheduling process as a Markov Decision Process (MDP), a Deep Q-Network (DQN) model based on Successor Feature (SF) is constructed, which is able to make decisions on scheduling at different time scales. The model decouples the states and rewards of tasks, and combines the decision attention module with the generalized policy improvement algorithm to propose a composite policy improvement algorithm (DA-GPI), which improves the flexibility and accuracy of SF-DQN decision making. The experiments are based on the Alibaba computer cluster dataset v2018, and the results show that the method outperforms the existing methods in terms of resource utilization, balance and scheduling response time.