Parallelization Strategies for DLRM Embedding Bag Operator on AMD CPUs

Deep learning recommendation models (DLRMs) are deployed extensively to support personalized recommendations and consume a large fraction of artificial intelligence (AI) cycles in modern datacenters with embedding stage being a critical component. Modern CPUs execute a lot of DLRM cycles because they are cost effective compared to GPUs and other accelerators. Our paper addresses key bottlenecks in accelerating the embedding stage on CPUs. Specifically, this work 1) explores novel threading schemes that parallelize embedding bag, 2) pushes the envelope on realized bandwidth by improving data reuse in caches, and 3) studies the impact of parallelization on load imbalance. The new embedding bag kernels have been prototyped in the ZenDNN software stack. When put together, our work on fourth generation EPYC processors achieve up to 9.9x improvement in embedding bag performance over state-of-the-art implementations, and improve realized bandwidth of up to 5.7x over DDR bandwidth.

Authors

Institutions

Publication Details

Journal
IEEE Micro
Published
2024-08-05
DOI
https://doi.org/10.1109/mm.2024.3423785
Citations
3
Primary Topic
Neural Networks and Applications
Type
article
Field-Weighted Citation Impact
1.04
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

Parallelization Strategies for DLRM Embedding Bag Operator on AMD CPUs

Jun-Liang Lin, Mahmut Kandemir, Meena Arunachalam, John Kalamatianos et al.
3 citations
IEEE Micro
Neural Networks and Applications
1.04
article

Parallelization Strategies for DLRM Embedding Bag Operator on AMD CPUs

Jun-Liang Lin, Mahmut Kandemir, Meena Arunachalam, John Kalamatianos, Varun Agrawal, Vinod Kathail, Ralph Wittig, Gulsum Gudukbay Akbulut, Saurabh Gupta, Elliott Delaye, Chita R. Das, Rajesh Vivekanandham, P. Gopalakrishnan, Ashish Sirasao, Satyaprakash Pareek, Rishabh Jain, Siddappa Karabannavar, Steve Reinhardt, Krishnakumar Nair, Avinash-Chandra Pandey
article en
3 citations

Abstract

Deep learning recommendation models (DLRMs) are deployed extensively to support personalized recommendations and consume a large fraction of artificial intelligence (AI) cycles in modern datacenters with embedding stage being a critical component. Modern CPUs execute a lot of DLRM cycles because they are cost effective compared to GPUs and other accelerators. Our paper addresses key bottlenecks in accelerating the embedding stage on CPUs. Specifically, this work 1) explores novel threading schemes that parallelize embedding bag, 2) pushes the envelope on realized bandwidth by improving data reuse in caches, and 3) studies the impact of parallelization on load imbalance. The new embedding bag kernels have been prototyped in the ZenDNN software stack. When put together, our work on fourth generation EPYC processors achieve up to 9.9x improvement in embedding bag performance over state-of-the-art implementations, and improve realized bandwidth of up to 5.7x over DDR bandwidth.

IEEE MicroVol. 44(6)
Pennsylvania State University (US), Advanced Micro Devices (Canada) (CA), Advanced Micro Devices (United States) (US)
Affordable and clean energy
Openalex Percentile: Top 20%
Neural Networks and Applications
1.04
DOISourceOpenAlex
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.