Hierarchical Power Co-Optimization and Management for LLM Chiplet Designs

The demand for efficient and high-performance hardware for large language models (LLMs) has driven the development of scalable chiplet design, which requires careful power optimization and management. This paper presents a co-optimization and management methodology for hierarchical power delivery of chiplet designs targeting LLM applications. To model LLM workload mapping and power delivery, we first build a scalable chiplet simulator, which demonstrates different power strategies have notable efficiency impact and require careful and thorough optimizations. We further develop a co-optimization framework ScalePoM for chiplet power management. Based on given LLM model and PPA requirements, ScalePoM can automatically explore the chiplet architecture and workload mapping for optimal hierarchical power delivery. Our co-optimization methodology is evaluated through two scaled LLM chiplets with different interconnect topologies, achieving an average of 45% and up to 62% energy saving for large language model inferences with various sparsity levels.