Faster and Memory-Efficient Training of Sequential Recommendation Models for Large Catalogs

Transformer-based sequential recommender systems are retrained frequently, yet their training is dominated by cross-entropy (CE) over item catalogs that can reach millions of items. Its logit tensor scales with catalog size × batch size × sequence length, forcing practitioners to trade these quantities against each other or use negative sampling. However, how to allocate memory among negatives, batch size, and sequence length remains an open question. We study this trade-off with nearly 1,000 SASRec training runs on six datasets and find that no single memory-related hyperparameter should be maximized in isolation. Scaling both the number of negatives and batch size outperforms maximizing only one, while the choice between full CE and CE with negative sampling is dataset-dependent. We further introduce a memory-efficient and GPU-optimized replacement for CE with negative sampling that avoids materializing the logit tensor. It speeds up training by up to 3.5× and reduces memory by 4–24× relative to PyTorch CE with negative sampling. Reallocating memory saved by our replacement and Cut Cross-Entropy, which we adapt to sequential recommender systems, into larger batches, longer sequences, and more negatives improves NDCG@10 by up to 54.7% under the same memory budget. We release the Triton kernel and a public dataset of all training runs at https://github.com/On-Point-RND/MemoryEfficientSRS .

Authors

Institutions

Publication Details

Journal
ACM Transactions on Information Systems
Published
2026-11-19
DOI
https://doi.org/10.1145/3862474
Citations
2
Primary Topic
Recommender Systems and Techniques
Type
article
Field-Weighted Citation Impact
2.31

Funders

Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Faster and Memory-Efficient Training of Sequential Recommendation Models for Large Catalogs

Alexey Zaytsev, Alexey Vasilev, Anna Volodkevich, Maxim Zhelnin et al.
2 citations
ACM Transactions on Information Systems
Recommender Systems and Techniques
2.31
article

Faster and Memory-Efficient Training of Sequential Recommendation Models for Large Catalogs

Alexey Zaytsev, Alexey Vasilev, Anna Volodkevich, Maxim Zhelnin, Egor Shvetsov, Petr Sokerin, Volkov Daniil, Redko Dmitry
article en
2 citations

Abstract

Transformer-based sequential recommender systems are retrained frequently, yet their training is dominated by cross-entropy (CE) over item catalogs that can reach millions of items. Its logit tensor scales with catalog size × batch size × sequence length, forcing practitioners to trade these quantities against each other or use negative sampling. However, how to allocate memory among negatives, batch size, and sequence length remains an open question. We study this trade-off with nearly 1,000 SASRec training runs on six datasets and find that no single memory-related hyperparameter should be maximized in isolation. Scaling both the number of negatives and batch size outperforms maximizing only one, while the choice between full CE and CE with negative sampling is dataset-dependent. We further introduce a memory-efficient and GPU-optimized replacement for CE with negative sampling that avoids materializing the logit tensor. It speeds up training by up to 3.5× and reduces memory by 4–24× relative to PyTorch CE with negative sampling. Reallocating memory saved by our replacement and Cut Cross-Entropy, which we adapt to sequential recommender systems, into larger batches, longer sequences, and more negatives improves NDCG@10 by up to 54.7% under the same memory budget. We release the Triton kernel and a public dataset of all training runs at https://github.com/On-Point-RND/MemoryEfficientSRS .

ACM Transactions on Information Systems
National Research University Higher School of Economics (RU), Lomonosov Moscow State University (RU)
Skolkovo Institute of Science and Technology, Ministero dello Sviluppo Economico
Openalex Percentile: Top 14%
Recommender Systems and Techniques
2.31
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon