Energy-Efficient GPU Resource Management for Generative AI on HPC Systems: A Roadmap Towards Cross-Layer Co-Design Approaches

Abstract Generative AI (GenAI) workloads are emerging as a major source of resource demand in high-performance computing (HPC) systems, creating new challenges for energy-efficient GPU management because of their extreme scale, memory and communication intensity, and highly dynamic training and inference behavior. Existing approaches remain fragmented across individual system layers, including hardware power control, runtime adaptation, and cluster scheduling, limiting their ability to deliver system-wide energy efficiency for large-scale GenAI deployment. This paper presents a structured review of 46 representative studies on energy-aware GPU resource management for GenAI on HPC platforms. The literature is synthesized through a cross-layer taxonomy spanning system layer, workload phase, decision type, evaluation scale, and energy-efficiency technique. The analysis shows that prior work has advanced telemetry-driven measurement, energy-performance prediction, scheduling and allocation, runtime power control, and architectural specialization, with several studies reporting substantial energy savings under specific workload and system conditions. At the same time, persistent gaps remain, including limited cross-layer coordination, weak support for token-level and phase-aware workload dynamics, and insufficient integration of carbon-aware objectives into resource management. To address these limitations, the paper proposes GenAI-XEnergy, a conceptual framework that unifies telemetry, prediction, orchestration, runtime control, and heterogeneous accelerator support within a closed-loop, cross-layer architecture. The review contributes a focused taxonomy, a comparative synthesis of existing techniques, an explicit analysis of open challenges, and a practical foundation for future research on sustainable, energy-proportional GenAI execution in HPC environments.

Authors

Institutions

Publication Details

Journal
Research Square
Published
2026-11-17
DOI
https://doi.org/10.21203/rs.3.rs-9666060/v1
Primary Topic
Parallel Computing and Optimization Techniques
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Energy-Efficient GPU Resource Management for Generative AI on HPC Systems: A Roadmap Towards Cross-Layer Co-Design Approaches

M. Usman Ashraf, Atha Shamas
3 links
Research Square
Parallel Computing and Optimization Techniques
preprint

Energy-Efficient GPU Resource Management for Generative AI on HPC Systems: A Roadmap Towards Cross-Layer Co-Design Approaches

M. Usman Ashraf, Atha Shamas
preprint en

Abstract

Abstract Generative AI (GenAI) workloads are emerging as a major source of resource demand in high-performance computing (HPC) systems, creating new challenges for energy-efficient GPU management because of their extreme scale, memory and communication intensity, and highly dynamic training and inference behavior. Existing approaches remain fragmented across individual system layers, including hardware power control, runtime adaptation, and cluster scheduling, limiting their ability to deliver system-wide energy efficiency for large-scale GenAI deployment. This paper presents a structured review of 46 representative studies on energy-aware GPU resource management for GenAI on HPC platforms. The literature is synthesized through a cross-layer taxonomy spanning system layer, workload phase, decision type, evaluation scale, and energy-efficiency technique. The analysis shows that prior work has advanced telemetry-driven measurement, energy-performance prediction, scheduling and allocation, runtime power control, and architectural specialization, with several studies reporting substantial energy savings under specific workload and system conditions. At the same time, persistent gaps remain, including limited cross-layer coordination, weak support for token-level and phase-aware workload dynamics, and insufficient integration of carbon-aware objectives into resource management. To address these limitations, the paper proposes GenAI-XEnergy, a conceptual framework that unifies telemetry, prediction, orchestration, runtime control, and heterogeneous accelerator support within a closed-loop, cross-layer architecture. The review contributes a focused taxonomy, a comparative synthesis of existing techniques, an explicit analysis of open challenges, and a practical foundation for future research on sustainable, energy-proportional GenAI execution in HPC environments.

Research Square
Government College Women University Sialkot (PK)
Affordable and clean energy
Parallel Computing and Optimization Techniques
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon