Ambient LLMs: Data-Efficient Supervised Fine-tuning via Temperature Matching
Collecting large-scale, diverse, and high-quality post-training datasets for autoregressive language models is challenging. Consequently, Supervised Fine-Tuning (SFT) is typically applied to datasets significantly smaller than those used for pre-training, which leads to overfitting and the catastrophic forgetting of the tails of the data distribution. To prevent this collapse, we introduce Ambient SFT, a principled framework for leveraging imperfect predictions to augment the fine-tuning set. In our method, a "frozen" version of the pre-trained model serves as a noisy teacher that the fine-tuned model must match at a coarse level. In particular, we require that the high-temperature versions of the conditional distributions produced by the post-training model should match the high-temperature versions of the pre-training ones. The temperature hyperparameter controls the level of trust in the pre-trained model, with the limit $T\to \infty$ reverting back to regular SFT and $T=1$ to knowledge distillation. We demonstrate that our approach outperforms standard SFT across domain adaptation, instruction-tuning, and reasoning, leading for up to $21\%$ average relative improvement to standard SFT.
Authors
- Giannis Daras
- Adam R. Klivans (ORCID: https://orcid.org/0000-0001-6960-2235)
- Constantinos Daskalakis (ORCID: https://orcid.org/0000-0002-5451-0490)
- Marios Papachristou (ORCID: https://orcid.org/0000-0002-1728-0729)
- Antonio B. Torralba (ORCID: https://orcid.org/0000-0003-4915-0256)
Publication Details
- Journal
- arXiv (Cornell University)
- Published
- 2026-11-17
- DOI
- https://doi.org/10.48550/arxiv.2611.23229
- Primary Topic
- Domain Adaptation and Few-Shot Learning
- Type
- preprint