An Empirical Study on the Power Consumption of LLMs with Different GPU Platforms

This paper researches on the power consumption of AIGC applications based on LLM with different parameter scales across different hardware platforms. Artificial Intelligence Generated Content (AIGC) represents a leading-edge application of AI technology, primarily driven by large language models (LLMs) and their associated technologies. The deployment of LLM typically relies on critical facilities with three layers, i.e., the hardware, model, and application layers. This empirical study aims to identify key factors in power consumption when a large model is serving in the inference stage, which will hint the insights for improving the energy efficiency of computational infrastructures. In the context of the "dual carbon" goals, i.e., carbon peaking and carbon neutrality, this study aims to find an effective way to reduce the energy cost of AIGC applications, thereby supporting sustainable AI development in industry.