KD4MT: A Survey of Knowledge Distillation for Machine Translation

Knowledge Distillation (KD) as a research area has gained a considerable traction in recent years as a compression tool to address challenges related to ever-larger models in NLP. Remarkably, Machine Translation (MT) offers a much more nuanced take on this narrative: in MT, KD also functions as a general-purpose knowledge transfer mechanism that shapes supervision and translation quality as well as efficiency. This survey synthesizes KD for MT (KD4MT) across 105 papers. We begin by introducing both MT and KD for non-experts, followed by an overview of the standard KD approaches relevant to MT applications. Subsequently, we categorize advances in the KD4MT literature based on (i) their methodological contributions and (ii) their practical applications. Our qualitative and quantitative analyses identify common trends in the field and highlight key research gaps as well as the absence of unified evaluation practice for KD methods in MT. We further provide practical guidelines for selecting a KD method in concrete settings and highlight potential risks associated with the application of KD to MT such as increased hallucination and bias amplification. Finally, we discuss the role of LLMs in re-shaping the KD4MT field. To support further research, we complement our survey with a publicly available database summarizing the main characteristics of the surveyed KD methods and a glossary of key terms.

Authors

Institutions

Publication Details

Journal
Computational Linguistics
Published
2026-11-21
DOI
https://doi.org/10.1162/coli.a.651
Citations
2
Primary Topic
Natural Language Processing Techniques
Type
article
Field-Weighted Citation Impact
1.86

Funders

Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

KD4MT: A Survey of Knowledge Distillation for Machine Translation

Yves Scherrer, Joseph Attieh, Timothee Mickus, Jörg Tiedemann et al.
2 citations
Computational Linguistics
Natural Language Processing Techniques
1.86
article

KD4MT: A Survey of Knowledge Distillation for Machine Translation

Yves Scherrer, Joseph Attieh, Timothee Mickus, Jörg Tiedemann, Ona de Gibert
article en
2 citations

Abstract

Knowledge Distillation (KD) as a research area has gained a considerable traction in recent years as a compression tool to address challenges related to ever-larger models in NLP. Remarkably, Machine Translation (MT) offers a much more nuanced take on this narrative: in MT, KD also functions as a general-purpose knowledge transfer mechanism that shapes supervision and translation quality as well as efficiency. This survey synthesizes KD for MT (KD4MT) across 105 papers. We begin by introducing both MT and KD for non-experts, followed by an overview of the standard KD approaches relevant to MT applications. Subsequently, we categorize advances in the KD4MT literature based on (i) their methodological contributions and (ii) their practical applications. Our qualitative and quantitative analyses identify common trends in the field and highlight key research gaps as well as the absence of unified evaluation practice for KD methods in MT. We further provide practical guidelines for selecting a KD method in concrete settings and highlight potential risks associated with the application of KD to MT such as increased hallucination and bias amplification. Finally, we discuss the role of LLMs in re-shaping the KD4MT field. To support further research, we complement our survey with a publicly available database summarizing the main characteristics of the surveyed KD methods and a glossary of key terms.

Computational Linguistics
University of Helsinki (FI), University of Oslo (NO)
Government of the United Kingdom, UK Research and Innovation, European Commission, HORIZON EUROPE Framework Programme
Openalex Percentile: Top 13%
Natural Language Processing Techniques
1.86
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon
KD4MT: A Survey of Knowledge Distillation for Machine Translation — Yves Scherrer, Joseph Attieh, et al. · Computational Linguistics (2026) | TGRS Research Map | TGRS