Characterizing and Efficiently Accelerating Multimodal Generation Model Inference

Generative artificial intelligence (AI) technology is revolutionizing the computing industry, posing new system design and optimization opportunities. In particular, AI’s ability to understand and respond in multiple modalities comes with significant system resource demands. To sustainably scale generative AI capabilities to billions of users in the world, inference must be fast and efficient. This article pinpoints key system design and optimization opportunities by characterizing a family of emerging multimodal generation models on real systems. Autoregressive token generation is a critical latency performance bottleneck, typically dominated by GPU idle time. In addition to memory-intensive attention across the generative AI models, linear operations constitute significant inference latency due to the feed forward networks in Transformer-based models. We demonstrate that state-of-the-art optimization levers, spanning from applications to system software and hardware, set a 3.88× better baseline.

Authors

Institutions

Publication Details

Journal
IEEE Micro
Published
2025-08-08
DOI
https://doi.org/10.1109/mm.2025.3596539
Citations
1
Primary Topic
Speech and dialogue systems
Type
article
Field-Weighted Citation Impact
2.48
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

Characterizing and Efficiently Accelerating Multimodal Generation Model Inference

Bilge Acun, Cristo M. Hernández Gómez, Kaushik Ram Sadagopan, Scott Yih et al.
1 citations
IEEE Micro
Speech and dialogue systems
2.48
article

Characterizing and Efficiently Accelerating Multimodal Generation Model Inference

Bilge Acun, Cristo M. Hernández Gómez, Kaushik Ram Sadagopan, Scott Yih, Carole-Jean Wu, Christian Puhrsch, Mostafa Elhoushi, Francisco Massa, Joel Schlosser, Can Balioglu, Samuel Hsia, Min-Jae Hwang, Juan Pino, Sravya Popuri, Daniel Haziza, Mingda Chen, Ram Pasunuru, Basil Hosmer, Jeremy Reizenstein, Linjian Ma, Anna Sun, Changhan Wang, Jiaqi Zhai, Xing Liu, Yejin Lee, Pedro Rodríguez, Alicia Golden, Leonid Shamis, Jeffrey K. S. Wan, Jimmy Isaacson, Jacob Kahn, Driss Guessous
article en
1 citations

Abstract

Generative artificial intelligence (AI) technology is revolutionizing the computing industry, posing new system design and optimization opportunities. In particular, AI’s ability to understand and respond in multiple modalities comes with significant system resource demands. To sustainably scale generative AI capabilities to billions of users in the world, inference must be fast and efficient. This article pinpoints key system design and optimization opportunities by characterizing a family of emerging multimodal generation models on real systems. Autoregressive token generation is a critical latency performance bottleneck, typically dominated by GPU idle time. In addition to memory-intensive attention across the generative AI models, linear operations constitute significant inference latency due to the feed forward networks in Transformer-based models. We demonstrate that state-of-the-art optimization levers, spanning from applications to system software and hardware, set a 3.88× better baseline.

IEEE MicroVol. 45(5)
National Society of Professional Engineers (US), Bellevue College (US), Institution of Civil Engineers (GB), Medieval Academy of America (US), Menlo School (US), Metro Transit (US), Visiting Nurse Association (US), Meta (United Kingdom) (GB), The Metropolitan Opera (United States) (US), Meta (United States) (US), ATUM (United States) (US), Metrica (United States) (US), Integrated Software (United States) (US), Metacomp Technologies (United States) (US), Seattle University (US)
Openalex Percentile: Top 9%
Speech and dialogue systems
2.48
DOISourceOpenAlex
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.