Software Performance Engineering for Foundation Model-Powered Software (FMware)

This keynote examines the transformative impact of Foundation Models (FMs), particularly Large Language Models (LLMs), on software development, emphasizing the critical role of Software Performance Engineering (SPE) in ensuring FM-powered software (FMware) achieves essential performance goals such as throughput and latency. With the LLM market projected to reach 36.1 billion by 2030 [3], addressing SPE challenges has become increasingly urgent. Drawing extensively from comprehensive literature surveys, industry-academia interactions, customer feedback, and practical experience detailed in [6]. This keynote identifies four critical SPE challenges throughout the FMware lifecycle, discusses current state-of-practice solutions, proposes future research directions, and introduces a vision for an innovative SLA-aware runtime system designed to enhance the performance and efficiency of FMware.