LinguaPrune: Language-Aware Mixed-Precision Quantization of a Mixture-of-Experts LLM for Kazakh and Russian

Mixture-of-Experts (MoE) language models hold most of their weights in experts, and quantizers calibrate those experts almost always on English text. We ask what this costs a low-resource language. We compress Qwen3-30B-A3B-Instruct-2507 for Kazakh and Russian by (i) measuring, layer by layer, which experts the router uses on Kazakh and Russian text, (ii) giving 4 bits to the experts that carry 90% of that traffic and 3 bits to the rest, and (iii) quantizing each expert with GPTQ, calibrated only on the Kazakh and Russian tokens routed to it. The result is a real file of 15.24 GB instead of 61.1 GB (4.0x smaller). Measured from the written file, perplexity on parallel Belebele passages grows by +1.3% for Kazakh, +0.4% for Russian and +3.2% for English; KazMMLU accuracy changes from 66.8% to 65.1% (Kazakh) and from 71.4% to 70.9% (Russian), differences we cannot distinguish from noise on our sample. At equal memory, choosing bits and calibration text by English instead of Kazakh and Russian raises the Kazakh perplexity loss from +0.9% to +14.3%. Against popular GGUF quantizations of the same model scored on the same exam, between 15 and 19 GB only our two files stay within 2% for both languages. We also report what does not work: removing experts erases facts, and 2-bit experts break the model although Kazakh perplexity stays within 2%. The study covers one model and one seed; code, per-question scores and notebooks that run on a free GPU are public. Code and per-question scores: https://github.com/s0nakh/linguapruneModels: kk+ru 4/3-bit, kk+ru 4-bit AI assistance: the author used Claude (Anthropic) as an additional tool; see the footnote on the first page.

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-11-16
DOI
https://doi.org/10.5281/zenodo.24281160
Primary Topic
Advanced Neural Network Applications
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

LinguaPrune: Language-Aware Mixed-Precision Quantization of a Mixture-of-Experts LLM for Kazakh and Russian

Sophia Akhmetova
3 links
Zenodo (CERN European Organization for Nuclear Research)
Advanced Neural Network Applications
preprint

LinguaPrune: Language-Aware Mixed-Precision Quantization of a Mixture-of-Experts LLM for Kazakh and Russian

Sophia Akhmetova
preprint en

Abstract

Mixture-of-Experts (MoE) language models hold most of their weights in experts, and quantizers calibrate those experts almost always on English text. We ask what this costs a low-resource language. We compress Qwen3-30B-A3B-Instruct-2507 for Kazakh and Russian by (i) measuring, layer by layer, which experts the router uses on Kazakh and Russian text, (ii) giving 4 bits to the experts that carry 90% of that traffic and 3 bits to the rest, and (iii) quantizing each expert with GPTQ, calibrated only on the Kazakh and Russian tokens routed to it. The result is a real file of 15.24 GB instead of 61.1 GB (4.0x smaller). Measured from the written file, perplexity on parallel Belebele passages grows by +1.3% for Kazakh, +0.4% for Russian and +3.2% for English; KazMMLU accuracy changes from 66.8% to 65.1% (Kazakh) and from 71.4% to 70.9% (Russian), differences we cannot distinguish from noise on our sample. At equal memory, choosing bits and calibration text by English instead of Kazakh and Russian raises the Kazakh perplexity loss from +0.9% to +14.3%. Against popular GGUF quantizations of the same model scored on the same exam, between 15 and 19 GB only our two files stay within 2% for both languages. We also report what does not work: removing experts erases facts, and 2-bit experts break the model although Kazakh perplexity stays within 2%. The study covers one model and one seed; code, per-question scores and notebooks that run on a free GPU are public. Code and per-question scores: https://github.com/s0nakh/linguapruneModels: kk+ru 4/3-bit, kk+ru 4-bit AI assistance: the author used Claude (Anthropic) as an additional tool; see the footnote on the first page.

Zenodo (CERN European Organization for Nuclear Research)
Advanced Neural Network Applications
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon