BottleMod: Modeling Data Flows and Tasks for Fast Bottleneck Analysis

In recent years, scientific workflows have become increasingly popular. However, their tasks are often seen as black boxes, making it difficult to optimize them or identify bottlenecks due to the complex relationships between tasks. Several factors impact task progress, including input data availability, computing power, data transfer speed and network connectivity. During task execution, resource requirements may change significantly. We propose a new method to model task requirements over their lifetime. Using these models, we predict resource consumption over time and execution duration based on a given allocation strategy with low overhead. This method enables computationally simple and fast performance predictions, including bottleneck analysis during workflow runtime. We derive a piecewise-defined bottleneck function from the discrete intersections of the task models' limiting functions. This allows us to predict potential performance gains when mitigating bottlenecks and aids in better resource allocation and workflow execution.