Serverless Workflow Benchmarking with HSMFlow and WfBench

The convergence of edge computing, big data analytics, and artificial intelligence (AI) with traditional scientific calculations is increasingly being adopted in high-performance computing (HPC) workflows. Workflow management systems are crucial for managing and orchestrating these complex computational tasks. However, it is difficult to identify patterns within the growing population of HPC workflows. Moreover, serverless has emerged as a novel computing paradigm, offering dynamic resource allocation, quick response time, fine-grained resource management and auto-scaling. In this tutorial, we present a framework to evaluate serverless executions of HPC scientific workflows. Our approach integrates a widely used traditional HPC workflow generator with an HPC serverless workflow management system to create benchmark suites of scientific workflows with diverse characteristics.