MoE--LoRA Methods for Multilingual Adaptation: A Systematic Empirical Study
Routed low-rank adapters offer a parameter-efficient means to improve the performance of large language models (LLMs) across tasks. Hence, it is assumed that they can also be used to improve the support of low-resource languages in LLMs, which currently serve English far better than the thousands of other languages spoken worldwide. However, this assumption has never been verified empirically under a shared multilingual evaluation protocol. We address exactly this gap by comparing eight low-rank adaptation (LoRA) methods on a frozen base model using 200 unique languages in more than 9{,}000 evaluation settings. Our results suggest that a reduced-scale search can identify viable configurations of LoRA methods but does not reliably predict their relative performance, as the top-ranking configuration is never best at scale. LoRA reliably improves language-model fit on translation data, consistently degrades reading comprehension, and benefits low-resource languages most. These findings suggest that LoRA methods should be regarded as targeted tools rather than general-purpose means for improving the multilingual capabilities of LLMs.
Authors
- Axel-Cyrille Ngonga Ngomo (ORCID: https://orcid.org/0000-0001-7112-3516)
- Nikit Srivastava (ORCID: https://orcid.org/0009-0004-5164-4911)
- René Speck (ORCID: https://orcid.org/0009-0001-0151-5538)
- Joel Dag
Publication Details
- Journal
- arXiv (Cornell University)
- Published
- 2026-11-17
- DOI
- https://doi.org/10.48550/arxiv.2611.23158
- Primary Topic
- Natural Language Processing Techniques
- Type
- preprint