Data Mixing for Large Language Models Pretraining: A Survey and Outlook

Large language models (LLMs) rely on pretraining on massive and highly heterogeneous corpora, where the composition of training data has a decisive impact on training efficiency and downstream generalization under realistic compute and data budget constraints. Unlike samplelevel data selection, data mixing optimizes domain-level sampling weights to allocate limited budgets more effectively. In recent years, a growing body of work has proposed principled data mixing methods for LLMs pretraining; however, the literature remains fragmented and lacks a dedicated, systematic survey. This paper provides a comprehensive review of data mixing for LLMs pretraining. We first formalize data mixture optimization as a bilevel problem on the probability simplex and clarify the role of data mixing in the pretraining pipeline, and briefly explain how existing methods make this bilevel formulation tractable in practice. We then introduce a fine-grained taxonomy that organizes existing methods along two main dimensions: static versus dynamic mixing. Static mixing is further categorized into rule-based and learning-based methods, while dynamic mixing is further grouped into adaptive and externally guided families. For each class of methods, we summarize representative approaches and analyze their characteristics, strengths, and limitations from a performance-cost trade-off perspective. Building on this analysis, we highlight key challenges that cut across methods, including limited transferability across data domains, optimization objectives, models, and validation sets, as well as unstandardized evaluation protocols and benchmarks, as well as the inherent tension between performance gains and cost control in learning-based methods. Finally, we outline several exploratory directions, including finer-grained domain partitioning and inverse data mixing, as well as pipeline-aware designs, aiming to provide conceptual and methodological insights for future research.