MLSynth: Towards Synthetic ML Traces

AI infrastructure continues to grow rapidly to meet the escalating demand for compute power required to train and inference increasingly capable models. This growth brings significant challenges in both the design and operation of ML pipelines. Exploring these challenges and evaluating potential solutions can be prohibitively expensive and time-consuming without effective simulation tools. This paper introduces MLSynth, a framework for synthesising ML workloads, which is essential for meaningful benchmarking of AI infrastructure. More specifically, MLSynth allows researchers to: (i) define a wide range of ML models with different parallelisation strategies, (ii) explore various sources of performance variability, and (iii) generate synthetic Chakra execution traces that can be used with existing simulation frameworks (e.g., ASTRA-Sim) to comprehensively model ML workloads.