Where 4-Bit Weight Quantization Breaks In-Context Interference Resolution: Layout, Operating Point and Format on a Paired-Associate Battery
Controlled evidence on 4-bit weight quantization and in-context interference is proactive-only, limited to one 4-bit format and behavioural. In 1.5–8B open-weight models we cross proactive and retroactive layouts, with a recency-counterbalanced control, with interference dose, list length and weight and KV-cache formats on item-paired recall trials, reading attention on the same trials. For Qwen2.5-1.5B, bitsandbytes NF4 damage is proactive-selective on 16- to 48-pair lists in accuracy and log-odds (accuracy selectivity +0.231 [+0.181, +0.281] on 16 pairs) and fades with list length, unresolved at 64 pairs. For Qwen2.5-7B no selectivity is identified: accuracy damage switches layout with list length while the log-odds layout difference does not change detectably. Equal-width NF4 and FP4 leave different signatures; INT8 weights and 8- and 4-bit KV caches stay near clean. On Qwen2.5 the interference-created attention shift sits on the damaged layout wherever attention was read and the accuracy selectivity is resolved, and predicts flips beyond the full-precision margin (held-out AUC gains +0.193 and +0.146). On Granite 4.2 3B at a matched operating point, load creates a layout contrast in the NF4 penalty (−0.37 [−0.68, −0.07]) because the retroactive cost shrinks under load; there NF4 moves attention away from the competitor, and the shift predicts flips only when intrusions alone count as flips. The selectivity does not transfer to a conflicting-document task; which interference 4-bit quantization harms must be measured on the deployment’s own material and operating point.
Authors
- Ya-Fen Yeh
- Guan-Yuan Chen (ORCID: https://orcid.org/0000-0003-3298-0624)
Institutions
- National Tsing Hua University (TW)
- North Carolina Exploring Cultural Heritage Online (US)
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-09-25
- DOI
- https://doi.org/10.5281/zenodo.22959852
- Primary Topic
- Radiation Effects in Electronics
- Type
- preprint