Stage-aware Bayesian-guided evolutionary algorithm for resource constrained serverless MapReduce scheduling

Serverless computing features instant elasticity and fine-grained billing. It has emerged as a prominent paradigm for big data processing such as log analysis and monitoring. However, scheduling MapReduce jobs under serverless resource constraints is challenging because the stateless nature of functions conflicts with MapReduce stage dependencies, which increases scheduling complexity. Existing research has not adequately captured these structural constraints, causing traditional evolutionary algorithms to explore the search space inefficiently. This study addresses the MapReduce scheduling problem under serverless resource constraints with the objective of minimizing the makespan. Accordingly, a stage-aware Bayesian-guided evolutionary algorithm (BGA) is proposed that, unlike traditional evolutionary algorithms, guides evolution through Bayesian learning from elite patterns. First, a mixed-integer programming model is formulated to unify temporal and precedence constraints. Second, a Bayesian probability model for assigning tasks to function instances guides evolutionary sampling using elite patterns, reducing inefficient search. Finally, dependency-aware crossover is integrated with adaptive variable neighborhood search to ensure feasibility and enhance local exploitation. Experimental results show that BGA consistently achieves the lowest makespan among compared methods, with Relative Percentage Deviation (RPD) remaining within 2.40%-3.92%.

Authors

Institutions

Publication Details

Journal
Tsinghua Science & Technology
Published
2026-11-18
DOI
https://doi.org/10.26599/tst.2026.9010102
Primary Topic
Cloud Computing and Resource Management
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Stage-aware Bayesian-guided evolutionary algorithm for resource constrained serverless MapReduce scheduling

Chuanzhi Chen, Hongyan Sang, Biao Zhang, Hengwei Guo et al.
2 links
Tsinghua Science & Technology
Cloud Computing and Resource Management
article

Stage-aware Bayesian-guided evolutionary algorithm for resource constrained serverless MapReduce scheduling

Chuanzhi Chen, Hongyan Sang, Biao Zhang, Hengwei Guo, Jinquan Zhang
article en

Abstract

Serverless computing features instant elasticity and fine-grained billing. It has emerged as a prominent paradigm for big data processing such as log analysis and monitoring. However, scheduling MapReduce jobs under serverless resource constraints is challenging because the stateless nature of functions conflicts with MapReduce stage dependencies, which increases scheduling complexity. Existing research has not adequately captured these structural constraints, causing traditional evolutionary algorithms to explore the search space inefficiently. This study addresses the MapReduce scheduling problem under serverless resource constraints with the objective of minimizing the makespan. Accordingly, a stage-aware Bayesian-guided evolutionary algorithm (BGA) is proposed that, unlike traditional evolutionary algorithms, guides evolution through Bayesian learning from elite patterns. First, a mixed-integer programming model is formulated to unify temporal and precedence constraints. Second, a Bayesian probability model for assigning tasks to function instances guides evolutionary sampling using elite patterns, reducing inefficient search. Finally, dependency-aware crossover is integrated with adaptive variable neighborhood search to ensure feasibility and enhance local exploitation. Experimental results show that BGA consistently achieves the lowest makespan among compared methods, with Relative Percentage Deviation (RPD) remaining within 2.40%-3.92%.

Tsinghua Science & Technology
Liaocheng University (CN)
Cloud Computing and Resource Management
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon