Quantization-Based Jailbreaking Vulnerability Analysis: A Study on Performance and Safety of the Llama3-8B-Instruct Model
This study systematically investigates how quantization affects the safety of large language models (LLMs), with a particular focus on jailbreaking vulnerabilities in multilingual and tense-shifted scenarios. Using Llama3-8B-Instruct as a representative open-source model, we evaluate 23 quantization levels across two languages (English/Korean) and three tenses (present/past/future). Our results reveal a non-linear relationship between quantization and safety: 4-bit quantization emerges as a practical "sweet spot," maintaining near-baseline performance while significantly reducing computational costs. However, substantial vulnerabilities persist—Korean prompts exhibit attack success rates 25.5 percentage points higher than their English counterparts, and past-tense transformations increase vulnerability by 39.3 percentage points. These findings highlight that current LLM safety mechanisms are often biased toward English and present-tense prompts, underscoring the need for new evaluation frameworks that account for linguistic and temporal diversity in quantized LLM deployment. Although this study has limitations, including its reliance on a single model, a single dataset, and automated evaluation method, it provides the first quantitative analysis of quantization-induced vulnerabilities in multilingual and tense-varied contexts.
Authors
- Jaesik Lee (ORCID: https://orcid.org/0009-0009-2389-9710)
Institutions
- National Information Society Agency (KR)
Publication Details
- Journal
- IEEE Access
- Published
- 2025-01-01
- DOI
- https://doi.org/10.1109/access.2025.3594287
- Citations
- 1
- Primary Topic
- Marine and Coastal Research
- Type
- article
- Field-Weighted Citation Impact
- 1.15