Carbon-Aware Workload Shifting for Mitigating Environmental Impact of Generative AI Models

The rapid proliferation of large language models (LLMs) has raised concerns about their substantial energy consumption during both training and inference phases, contributing to environmental concerns. In particular, inference activities constitute a significant portion of carbon emissions, exacerbated by the expansive user base of LLM applications. Furthermore, the carbon intensity of data centers varies widely across different regions, suggesting an opportunity for carbon-aware workload shifting to mitigate emissions. In this study, we investigate the efficacy of such a strategy by examining three prominent large generative AI models: Mistral, Vicuna, and Stable Diffusion. We conducted comprehensive experiments to measure power consumption, runtime, and carbon emission of various prompts across five regions in the U.S. and Europe. These empirical data were then utilized to construct a simulation to assess the potential carbon savings achievable through carbon-aware workload shifting. Our results demonstrate that redirecting requests to regions with lower carbon intensities could yield substantial reductions in emissions, with a projected decrease of over 1,652 pounds on a single day per 10 million requests. This research demonstrates the great potential of reducing the environmental impact of AI and paves the way for sustainable deployment of large generative AI models.