Energy-Efficient GPU Resource Management for Generative AI on HPC Systems: A Roadmap Towards Cross-Layer Co-Design Approaches
Abstract Generative AI (GenAI) workloads are emerging as a major source of resource demand in high-performance computing (HPC) systems, creating new challenges for energy-efficient GPU management because of their extreme scale, memory and communication intensity, and highly dynamic training and inference behavior. Existing approaches remain fragmented across individual system layers, including hardware power control, runtime adaptation, and cluster scheduling, limiting their ability to deliver system-wide energy efficiency for large-scale GenAI deployment. This paper presents a structured review of 46 representative studies on energy-aware GPU resource management for GenAI on HPC platforms. The literature is synthesized through a cross-layer taxonomy spanning system layer, workload phase, decision type, evaluation scale, and energy-efficiency technique. The analysis shows that prior work has advanced telemetry-driven measurement, energy-performance prediction, scheduling and allocation, runtime power control, and architectural specialization, with several studies reporting substantial energy savings under specific workload and system conditions. At the same time, persistent gaps remain, including limited cross-layer coordination, weak support for token-level and phase-aware workload dynamics, and insufficient integration of carbon-aware objectives into resource management. To address these limitations, the paper proposes GenAI-XEnergy, a conceptual framework that unifies telemetry, prediction, orchestration, runtime control, and heterogeneous accelerator support within a closed-loop, cross-layer architecture. The review contributes a focused taxonomy, a comparative synthesis of existing techniques, an explicit analysis of open challenges, and a practical foundation for future research on sustainable, energy-proportional GenAI execution in HPC environments.
Authors
- M. Usman Ashraf
- Atha Shamas
Institutions
- Government College Women University Sialkot (PK)
Publication Details
- Journal
- Research Square
- Published
- 2026-11-17
- DOI
- https://doi.org/10.21203/rs.3.rs-9666060/v1
- Primary Topic
- Parallel Computing and Optimization Techniques
- Type
- preprint