Transformer-Enhanced DQN Approach for Energy and Cost-Efficient Large-Scale Dynamic Workflow Scheduling in Heterogeneous Environment

In a heterogeneous workflow environment, the uncertainty of task execution times, dynamic resource changes, and task dependencies’ evolution pose significant scheduling challenges. This article investigates how to make intelligent and adaptive scheduling decisions in a constantly changing heterogeneous cloud environment. We propose a novel scheduling approach, transformer-enhanced DQN (T-DQN) that combines the strengths of reinforcement learning (RL) and the Transformer model into a hybrid strategy. This method leverages the ability of RL to handle uncertainty and dynamics in the decision-making process while integrating the advantages of the Transformer model in dealing with long sequences and complex relationships. Our experimental evaluation shows that the T-DQN algorithm outperforms existing algorithms consistently in real-world workflow, dynamic scenarios, and high-load environments. T-DQN reduces makespan by up to 13.66%, improves energy by about 16.65%, and improves cost by 44.72% compared to the six other approaches. This performance is particularly significant in high-load environments, where T-DQN’s adaptability and scalability minimize failure rates and optimize resource management, affirming its suitability as a robust solution to complex cloud computing challenges.