Adding a Thai N-gram Engram to Qwen3-ASR-0.6B: effects on transcription error, memory, and latency
Technical report, version 1.0.0, in English and Thai, deposited as a preprint. Both editions are included as PDF, in full, one file per language. The report asks whether a token-level N-gram memory (Engram) reduces Thai transcription errors when added to a frozen Qwen3-ASR-0.6B, and what three numerical precisions cost in quality, model-tensor storage, peak allocated VRAM, and latency. It is written for a reader who is new to ASR terminology: the terms that carry the argument are defined where they are used. Contents: five chapters covering the problem and scope, the background and model design, the methodology, the results, and the conclusions, followed by acknowledgements, a reproducibility-artifact appendix, and the bibliography. Findings, with the limits that apply to each: - Engram step 750 in BF16 reduced corpus character error rate from 17.8051% to 9.0448% on a fixed 300-clip Thai evaluation set, with character-level micro F1 rising from 86.3235% to 92.9302%. This is one fixed sample from one offset of one corpus, not the full dataset, and no confidence intervals were computed. - Quantization cost quality in this setup. NF4 raised CER to 28.7086% for the baseline and 20.9178% for Engram; LLM.int8 raised it to 18.8901% and 10.8413%. Engram NF4 remained 3.1128 CER percentage points above the BF16 baseline, which is a difference in CER and not a 3% loss of accuracy. - Smaller weights were not faster. Both quantized variants ran slower per clip than their BF16 counterpart on this GPU and backend. NF4 reduced model-tensor storage by about 62% and LLM.int8 by about 29%, but both increased peak allocated VRAM. - The two quantizers do not touch the same parameters: NF4 applies to Linear and Embedding modules, LLM.int8 to Linear modules only, leaving embeddings in BF16. This is a limitation of the comparison and the report says so. The report is explicit that these are measurements of one ROCm/bitsandbytes configuration on one GPU, not general performance claims, and that the Engram training loop is not part of the deposit. Quantized GGUF inference is out of scope and no number comes from llama.cpp. The Thai speech data comes from the Porjai-Thai-voice-dataset-central corpus (CMKL University, CC BY-SA 4.0), which is not redistributed; the 300 clips used are identified only by offset and by a frozen reference list. The base model is Qwen3-ASR-0.6B from the Qwen Team.
Authors
- Thanabodee Nammungkun (ORCID: https://orcid.org/0009-0004-9410-9839)
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-09-24
- DOI
- https://doi.org/10.5281/zenodo.22933543
- Primary Topic
- Natural Language Processing Techniques
- Type
- preprint