chem v5: a QLoRA chemistry adapter for Qwen3-8B. Sealed-test evidence of a +5.0-point gain on multiple-choice chemistry, and an honest account of a failed development gate
chem v5 is a QLoRA adapter (rank 16, about 43.6 M trainable parameters) for Qwen3-8B (Apache-2.0), trained for one epoch on 11,839 permissively licensed chemistry examples on a single 12 GB laptop GPU. Base and adapter were compared zero-shot, non-thinking, with letter-probability scoring and 4-bit NF4 inference on three held-out multiple-choice chemistry benchmarks, with paired bootstrap 95% confidence intervals (10,000 resamples). Sealed test (2,529 questions, run once on 17 November 2026): pooled accuracy 47.61% → 52.63% (+5.02 points, 95% CI +3.24 to +6.76); MMLU chemistry 63.95 → 67.81 (+3.86, CI −0.43 to +8.15), ChemBench MCQ 54.82 → 58.22 (+3.39, CI +1.13 to +5.72), MMLU-Pro chemistry 29.13 → 37.58 (+8.45, CI +5.17 to +11.73); 328 items gained, 201 lost; pre-specified gate passed. Development split (1,083 questions), same rule: pooled +1.20 points (CI −1.29 to +3.79), ChemBench −1.40; gate failed. Both results are reported side by side, together with a crash and partial re-run during the sealed test and a discussion of the dev/test gap. Italian and English versions are included. Author: Giovanni Del Papa, Independent researcher, Italy. chem v5 è un adattatore QLoRA (rango 16, circa 43,6 M parametri addestrabili) per Qwen3-8B (Apache-2.0), addestrato per un'epoca su 11.839 esempi di chimica con licenze permissive su una sola GPU da portatile da 12 GB. Test sigillato (2.529 domande, 17 novembre 2026): accuratezza complessiva 47,61% → 52,63% (+5,02 punti, IC 95% da +3,24 a +6,76), gate superato. Split di sviluppo (1.083 domande): +1,20 punti (IC da −1,29 a +3,79), gate non superato. Entrambi i risultati sono riportati affiancati. Version 1.1 removes a leftover draft line under the title. There are no content changes.
Authors
- Giovanni Del Papa
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-11-17
- DOI
- https://doi.org/10.5281/zenodo.24326181
- Primary Topic
- Domain Adaptation and Few-Shot Learning
- Type
- preprint