QoS aware static workflow scheduling using Q-learning in cloud environment

Cloud computing is an emerging area of research in the field of computer science. It can be applied to various domains such as computational physics, bioinformatics, and pharmaceutical sciences. Workflow scheduling is a critical problem in cloud computing because the execution of scientific workflows involves precedence constraints, heterogeneous virtual machines, and competing resource requirements. Efficient task prioritization and resource allocation are therefore essential for reducing workflow execution time and improving overall resource utilization. This study proposes a hybrid framework for deterministic workflow scheduling that combines an incidence-matrix-based heuristic strategy with Q-learning. The heuristic component exploits the dependency relationships among workflow tasks to derive task priorities, while the Q-learning component adaptively refines the scheduling decisions for effective task-to-resource allocation. The primary objective of the proposed framework is to minimize workflow makespan while maintaining balanced and efficient utilization of available virtual machines. Its performance is evaluated using established scientific workflow benchmarks, including CyberShake and Epigenomics are compared with the HEFT, FCFS, PETS, QL-HEFT, and Modified Min-Min algorithms. The evaluation considers makespan, speedup, efficiency, load balancing, and resource utilization as performance measures. Experimental results demonstrate that the proposed framework achieves lower makespan than the compared scheduling approaches across the evaluated workflows. For the CyberShake workflow, the proposed method provides an average makespan reduction of approximately 11.84%, while for the Epigenomics workflow, it provides an average makespan reduction of approximately 7.07% to 16.45%. Furthermore, one-way ANOVA confirms that the differences among the evaluated scheduling approaches are statistically significant. These findings indicate that integrating dependency-aware heuristic prioritization with Q-learning can improve the effectiveness of workflow scheduling and provide a suitable approach for efficient resource allocation in cloud computing environments.