Unveiling Environmental Impacts of Large Language Model Serving: A Functional Unit View

Large language models (LLMs) offer powerful capabilities but come with significant environmental impact, particularly in carbon emissions.Existing studies benchmark carbon emissions but lack a standardized basis for comparison across different model configurations.To address this, we introduce the concept of functional unit (FU) as a standardized basis and develop FUEL, the first FU-based framework for evaluating LLM serving's environmental impact.Through three case studies, we uncover key insights and trade-offs in reducing carbon emissions by optimizing model size, quantization strategy, and hardware choice, paving the way for more sustainable LLM serving.