EMU-LLM: Emulators for Performance Evaluation of LLM-based Applications

With the advent of Large Language Models (LLMs) in modern web applications, rigorous performance testing has become essential to assess application's speed, stability, and resource usage under diverse workload conditions. While third-party APIs enable rapid integration and scaling in LLM-based applications, scalability testing with these APIs often incurs high costs and adds complexity throughout the development cycle. To address these challenges, we propose EMU-LLM, a framework that automatically selects and integrates emulators to replace third-party APIs while preserving LLM behavior. This emulator-centric approach offers a cost-effective solution for performance evaluation and system testing of LLM-based web applications. Our framework enhances the robustness and efficiency of applications and highlights promising directions for future research. This paper aims to guide researchers, developers, and testers on the significance of emulators in optimizing LLM performance and fostering growth.