Precision vs. Pragmatics: Benchmarking Post-Training Quantization Trade-offs in Sovereign Korean Large Language Models
The rapid deployment of Sovereign Large Language Models (LLMs) in South Korea requires balancing high computational efficiency with strict preservation of linguistic pragmatics, particularly hierarchical honorifics (Jondaetmal) and cultural context. While 4-bit Post-Training Quantization (PTQ) significantly reduces memory footprints for edge deployment on Neural Processing Units (NPUs), its impact on culturally sensitive Korean generation remains underexplored. In this study, we empirically benchmark the memory efficiency and pragmatic fidelity of 4-bit NormalFloat (NF4) quantization applied to a 2.8-billion-parameter Korean sovereign LLM (EEVE-Korean-Instruct-2.8B-v1.0) on an NVIDIA T4 GPU. Our results demonstrate that NF4 quantization reduces model memory footprint by 67.2% (from 5.64 GB in FP16 to 1.85 GB in INT4) while preserving grammatical honorific structure in formal workplace communication.
Authors
- Radhika Maliye
Institutions
- Centre for Artificial Intelligence and Robotics (IN)
- Artificial Intelligence in Medicine (Canada) (CA)
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-11-16
- DOI
- https://doi.org/10.5281/zenodo.24285712
- Primary Topic
- Advanced Neural Network Applications
- Type
- preprint