Findings of the WMT26 Multilingual Instruction Shared Task: Small Models Are Not Yet Polyglots
We present the findings of the WMT26 Multilingual Instruction Shared Task (MIST), evalu- ating small (≤10B parameter) open-weight language models across 24 languages on three subtasks: context-based QA, cross-lingual summarization, and open-ended generation. Submissions from eight teams explore fine-tuning, knowledge distillation, and prompt engineering. Our findings demonstrate that small models are not yet polyglots: while leading systems perform well on high-resource languages, per- formance drops sharply on low-resource varieties, and comprehension degrades signifi- cantly when transferring across non-English language pairs. In open-ended generation, human evaluation and automatic rule-based verification exhibit strong overall rank correlation (ρ=0.79) but diverge locally at the top: human evaluators favor distilled pipelines with superior fluency and cross-lingual equity, whereas automatic verifiers reward concise promptengineered systems. Furthermore, multi-stage pipelines suffer from prompt-language inertia on cross-lingual instructions, and summariza- tion exhibits severe English leakage unless penalized by language gating. Finally, while baseline fluency is largely achieved by modern small models, multi-constraint instruction following and unanswerable question refusal remain the primary performance bottlenecks. We release all test sets, system outputs, human judgments, and evaluation code under a permissive license.
Authors
- Beatrice Savoldi (ORCID: https://orcid.org/0000-0002-3061-8317)
- Mariya Shmatova
- Patrícia Schmidtová (ORCID: https://orcid.org/0009-0008-5516-798X)
- Seth Aycock
- Evfrosiniya Zerminova
- Tom Kocmi (ORCID: https://orcid.org/0000-0002-7993-9859)
- Sara Papi (ORCID: https://orcid.org/0000-0002-4494-8886)
- Ekaterina Artemova (ORCID: https://orcid.org/0000-0001-8689-7514)
- Niyati Bafna
- Vilém Zouhar (ORCID: https://orcid.org/0000-0001-9874-2069)
- Danni Liu (ORCID: https://orcid.org/0000-0001-5419-1963)
- Maike Züfle
- Hanuman Sidh
- Christof Monz
- Barry Haddow
- Manmeet Kaur
- Philipp Koehn
- Nam Luu
- Shobhit Banga
- Pinzhen Chen
Publication Details
- Journal
- KITopen
- Published
- 2026-11-04
- DOI
- https://doi.org/10.5445/ir/1000198780
- Primary Topic
- Natural Language Processing Techniques
- Type
- article
- Field-Weighted Citation Impact
- 0.00