Reducing Energy Bloat in Large Model Training
Training large AI models on numerous GPUs consumes a massive amount of energy, making power delivery one of the largest limiting factors in building and operating datacenters for AI workloads. However, we observe that not all energy consumed during training directly contributes to end-to-end throughput; a significant portion can be removed without slowing down training. We call this portion energy bloat.
Authors
- Nikhil Bansal (ORCID: https://orcid.org/0000-0002-6290-0894)
- Mosharaf Chowdhury (ORCID: https://orcid.org/0000-0003-0884-6740)
- Luoxi Meng
- Jae-Won Chung (ORCID: https://orcid.org/0000-0001-5924-3427)
- Insu Jang (ORCID: https://orcid.org/0009-0007-5206-2333)
- Yile Gu
Institutions
- University of Washington (US)
- University of Michigan (US)
- University of California San Diego (US)
Publication Details
- Published
- 2024-11-04
- DOI
- https://doi.org/10.1145/3694715.3695970
- Citations
- 10
- Primary Topic
- Advanced Neural Network Applications
- Type
- article
- Field-Weighted Citation Impact
- 2.49
Funders
- National Science Foundation