Sustainable LLM Serving: Environmental Implications, Challenges, and Opportunities : Invited Paper

Large language models (LLMs) have been widely used for their ability to handle complex natural language tasks with high accuracy. The lifecycle of LLMs development and deployment encompasses both training and serving phases. Although training takes months and consumes significant amounts of energy, recent studies show that the energy consumption of LLM serving has now surpassed that of training, leading to significant environmental impacts, especially in terms of carbon footprints. While much prior work has focused on improving LLM performance, the specific challenge of reducing the carbon footprint of LLM serving has been largely overlooked. This paper identifies key challenges and outlines research directions for making LLM serving more sustainable, aiming to inspire further environmentally responsible advancements in the field.