Alleviating All-to-All Communication for Deep Learning Recommendation Model Inference
Massive DLRMs require large-scale multi-node systems for distributed training and inference, thus suffering from the all-to-all communication bottleneck. We propose an architecture, EmbedSwitch, that offloads the cache function of the embedding table vectors to a programmable switch, to overcome this bottleneck and provide switch-level response latency for embedding table vector requests.
Authors
- Jingpu Duan (ORCID: https://orcid.org/0000-0001-7507-2197)
- Shuo Liu (ORCID: https://orcid.org/0000-0002-3546-4070)
- L P Chen
- Wenfei Wu (ORCID: https://orcid.org/0000-0002-1357-3137)
- Songjun Huang (ORCID: https://orcid.org/0009-0005-9900-0530)
- Xu Chen (ORCID: https://orcid.org/0000-0001-9943-6020)
- Xiaoxi Zhang (ORCID: https://orcid.org/0000-0003-0751-2773)
- Yihong Li (ORCID: https://orcid.org/0000-0002-0622-1969)
Institutions
- Sun Yat-sen University (CN)
- Peking University (CN)
- Huawei Technologies (China) (CN)
- Peng Cheng Laboratory (CN)
Publication Details
- Published
- 2024-08-09
- DOI
- https://doi.org/10.1145/3677333.3678267
- Primary Topic
- Recommender Systems and Techniques
- Type
- article
- Field-Weighted Citation Impact
- 0.00