FNOPerf: A Robust Empirical Model for Predicting LLM Performance

Performance models are often used for quick evaluation of design choices, thereby making it one of the most exploited tools available to software and system designers. Managing critical computing resources becomes uphill task when it comes to training Large Language Models (LLMs) due to the sheer volume of data and compute power required to handle the job in a reasonable time. In this paper, we attempt to augment a critical piece in the developer tool-chain by designing FNOPerf, an empirical performance model based on Fourier Neural Operator (FNO), derived from experimental data collected over a modest size of 1500 data samples. Our experimental results with state-of-the-art benchmarks show that FNOPerf provide high accuracy to predict runtime performance with a MAPE of 4.12%.