Greek versus English Degradation Under Consumer GGUF Quantization

Local inference of Large Language Models (LLMs) depends on quantization to trade accuracy for efficiency on consumer hardware. This trade-off is usually measured in English, raising the question of whether speakers of other languages (like Greek) suffer more with the same file — and if so, by how much, and with what mechanism of distortion. No prior work has characterized the consumer GGUF k-quant ladder with sub-4B models and distributional metrics. I evaluate Greek and English across five k-quant levels (Q6_K to Q2_K) on base Llama 3.2 3B and 1B, measuring mean and median per-token KL divergence and top-1 token agreement from a Q8_0 reference over a parallel FLORES+ corpus. I find that the mean Greek-English KL divergence ratio never falls below 1.35 on either model; damage migrates from tail to typical tokens as precision decreases; and the practical floor of usage is Q4_K_M on 3B and Q5_K_M on 1B, while Q2_K is unusable.

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-08-28
DOI
https://doi.org/10.5281/zenodo.22145680
Primary Topic
Natural Language Processing Techniques
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
preprint

Greek versus English Degradation Under Consumer GGUF Quantization

William Alexakis
Zenodo (CERN European Organization for Nuclear Research)
Natural Language Processing Techniques
preprint

Greek versus English Degradation Under Consumer GGUF Quantization

William Alexakis
preprint en

Abstract

Local inference of Large Language Models (LLMs) depends on quantization to trade accuracy for efficiency on consumer hardware. This trade-off is usually measured in English, raising the question of whether speakers of other languages (like Greek) suffer more with the same file — and if so, by how much, and with what mechanism of distortion. No prior work has characterized the consumer GGUF k-quant ladder with sub-4B models and distributional metrics. I evaluate Greek and English across five k-quant levels (Q6_K to Q2_K) on base Llama 3.2 3B and 1B, measuring mean and median per-token KL divergence and top-1 token agreement from a Q8_0 reference over a parallel FLORES+ corpus. I find that the mean Greek-English KL divergence ratio never falls below 1.35 on either model; damage migrates from tail to typical tokens as precision decreases; and the practical floor of usage is Q4_K_M on 3B and Q5_K_M on 1B, while Q2_K is unusable.

Zenodo (CERN European Organization for Nuclear Research)
Natural Language Processing Techniques
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.