Precision vs. Pragmatics: Benchmarking Post-Training Quantization Trade-offs in Sovereign Korean Large Language Models

The rapid deployment of Sovereign Large Language Models (LLMs) in South Korea requires balancing high computational efficiency with strict preservation of linguistic pragmatics, particularly hierarchical honorifics (Jondaetmal) and cultural context. While 4-bit Post-Training Quantization (PTQ) significantly reduces memory footprints for edge deployment on Neural Processing Units (NPUs), its impact on culturally sensitive Korean generation remains underexplored. In this study, we empirically benchmark the memory efficiency and pragmatic fidelity of 4-bit NormalFloat (NF4) quantization applied to a 2.8-billion-parameter Korean sovereign LLM (EEVE-Korean-Instruct-2.8B-v1.0) on an NVIDIA T4 GPU. Our results demonstrate that NF4 quantization reduces model memory footprint by 67.2% (from 5.64 GB in FP16 to 1.85 GB in INT4) while preserving grammatical honorific structure in formal workplace communication.

Authors

Institutions

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-11-16
DOI
https://doi.org/10.5281/zenodo.24285712
Primary Topic
Advanced Neural Network Applications
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Precision vs. Pragmatics: Benchmarking Post-Training Quantization Trade-offs in Sovereign Korean Large Language Models

Radhika Maliye
2 links
Zenodo (CERN European Organization for Nuclear Research)
Advanced Neural Network Applications
preprint

Precision vs. Pragmatics: Benchmarking Post-Training Quantization Trade-offs in Sovereign Korean Large Language Models

Radhika Maliye
preprint en

Abstract

The rapid deployment of Sovereign Large Language Models (LLMs) in South Korea requires balancing high computational efficiency with strict preservation of linguistic pragmatics, particularly hierarchical honorifics (Jondaetmal) and cultural context. While 4-bit Post-Training Quantization (PTQ) significantly reduces memory footprints for edge deployment on Neural Processing Units (NPUs), its impact on culturally sensitive Korean generation remains underexplored. In this study, we empirically benchmark the memory efficiency and pragmatic fidelity of 4-bit NormalFloat (NF4) quantization applied to a 2.8-billion-parameter Korean sovereign LLM (EEVE-Korean-Instruct-2.8B-v1.0) on an NVIDIA T4 GPU. Our results demonstrate that NF4 quantization reduces model memory footprint by 67.2% (from 5.64 GB in FP16 to 1.85 GB in INT4) while preserving grammatical honorific structure in formal workplace communication.

Zenodo (CERN European Organization for Nuclear Research)
Centre for Artificial Intelligence and Robotics (IN), Artificial Intelligence in Medicine (Canada) (CA)
Advanced Neural Network Applications
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon