External Large Foundation Model: How to Efficiently Serve Trillions of Parameters for Online Ads Recommendation

Ads recommendation is a prominent service of online advertising systems and has been actively studied. Recent studies indicate that scaling-up and advanced design of the recommendation model can bring significant performance improvement. However, with a larger model scale, such prior studies have a significantly increasing gap from industry as they often neglect two fundamental challenges in industrial-scale applications. First, training and inference budgets are restricted for the model to be served, exceeding which may incur latency and impair user experience. Second, large-volume data arrive in a streaming mode with data distributions dynamically shifting, as new users/ads join and existing users/ads leave the system. We propose the External Large Foundation Model (ExFM) framework to address the overlooked challenges. Specifically, we develop external distillation and a data augmentation system (DAS) to control the computational cost of training/inference while maintaining high performance. We design the teacher in a way like a foundation model (FM) that can serve multiple students as vertical models (VMs) to amortize its building cost. We propose Auxiliary Head and Student Adapter to mitigate the data distribution gap between FM and VMs caused by the streaming data issue. Comprehensive experiments on internal industrial-scale applications and public datasets demonstrate significant performance gain by ExFM.

Authors

Institutions

Publication Details

Published
2025-05-08
DOI
https://doi.org/10.1145/3701716.3715223
Citations
6
Primary Topic
Recommender Systems and Techniques
Type
article
Field-Weighted Citation Impact
19.48
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

External Large Foundation Model: How to Efficiently Serve Trillions of Parameters for Online Ads Recommendation

Shuyu Xu, Ellie Wen, Zhengyu Zhang, Shali Jiang et al.
6 citations
Recommender Systems and Techniques
19.48
article

External Large Foundation Model: How to Efficiently Serve Trillions of Parameters for Online Ads Recommendation

Shuyu Xu, Ellie Wen, Zhengyu Zhang, Shali Jiang, Fan Yang, Hansey Chen, Laming Chen, Mengyue Hang, Qiuling Suo, Jiyan Yang, Weilin Zhang, Wenlin Chen, Qianru Li, Yasmine Badr, Santanu Kolay, Zhichen Zeng, Xingliang Huang, Rong Jin, Bo Long, Kai Wang, Wenjing Lu, Chen Die Yang, Mingfu Liang, Wenjun Wang, Huayu Li, Xi Liu, Qi Nie, Qinghai Zhou, Zheng Hua, Rui Zhang, Zhiyuan Li, Song Zhou, Boyang Liu, Shiquan Wang, Xiaozhen Xia, Evelyn Lyu, Jason Rudy
article en
6 citations

Abstract

Ads recommendation is a prominent service of online advertising systems and has been actively studied. Recent studies indicate that scaling-up and advanced design of the recommendation model can bring significant performance improvement. However, with a larger model scale, such prior studies have a significantly increasing gap from industry as they often neglect two fundamental challenges in industrial-scale applications. First, training and inference budgets are restricted for the model to be served, exceeding which may incur latency and impair user experience. Second, large-volume data arrive in a streaming mode with data distributions dynamically shifting, as new users/ads join and existing users/ads leave the system. We propose the External Large Foundation Model (ExFM) framework to address the overlooked challenges. Specifically, we develop external distillation and a data augmentation system (DAS) to control the computational cost of training/inference while maintaining high performance. We design the teacher in a way like a foundation model (FM) that can serve multiple students as vertical models (VMs) to amortize its building cost. We propose Auxiliary Head and Student Adapter to mitigate the data distribution gap between FM and VMs caused by the streaming data issue. Comprehensive experiments on internal industrial-scale applications and public datasets demonstrate significant performance gain by ExFM.

University of Illinois Urbana-Champaign (US), Meta (United States) (US)
Openalex Percentile: Top 1%
Recommender Systems and Techniques
19.48
DOIOpen Access PDFOpenAlex
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.