chem v5: a QLoRA chemistry adapter for Qwen3-8B. Sealed-test evidence of a +5.0-point gain on multiple-choice chemistry, and an honest account of a failed development gate

chem v5 is a QLoRA adapter (rank 16, about 43.6 M trainable parameters) for Qwen3-8B (Apache-2.0), trained for one epoch on 11,839 permissively licensed chemistry examples on a single 12 GB laptop GPU. Base and adapter were compared zero-shot, non-thinking, with letter-probability scoring and 4-bit NF4 inference on three held-out multiple-choice chemistry benchmarks, with paired bootstrap 95% confidence intervals (10,000 resamples). Sealed test (2,529 questions, run once on 17 November 2026): pooled accuracy 47.61% → 52.63% (+5.02 points, 95% CI +3.24 to +6.76); MMLU chemistry 63.95 → 67.81 (+3.86, CI −0.43 to +8.15), ChemBench MCQ 54.82 → 58.22 (+3.39, CI +1.13 to +5.72), MMLU-Pro chemistry 29.13 → 37.58 (+8.45, CI +5.17 to +11.73); 328 items gained, 201 lost; pre-specified gate passed. Development split (1,083 questions), same rule: pooled +1.20 points (CI −1.29 to +3.79), ChemBench −1.40; gate failed. Both results are reported side by side, together with a crash and partial re-run during the sealed test and a discussion of the dev/test gap. Italian and English versions are included. Author: Giovanni Del Papa, Independent researcher, Italy. chem v5 è un adattatore QLoRA (rango 16, circa 43,6 M parametri addestrabili) per Qwen3-8B (Apache-2.0), addestrato per un'epoca su 11.839 esempi di chimica con licenze permissive su una sola GPU da portatile da 12 GB. Test sigillato (2.529 domande, 17 novembre 2026): accuratezza complessiva 47,61% → 52,63% (+5,02 punti, IC 95% da +3,24 a +6,76), gate superato. Split di sviluppo (1.083 domande): +1,20 punti (IC da −1,29 a +3,79), gate non superato. Entrambi i risultati sono riportati affiancati. Version 1.1 removes a leftover draft line under the title. There are no content changes.

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-11-17
DOI
https://doi.org/10.5281/zenodo.24326181
Primary Topic
Domain Adaptation and Few-Shot Learning
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

chem v5: a QLoRA chemistry adapter for Qwen3-8B. Sealed-test evidence of a +5.0-point gain on multiple-choice chemistry, and an honest account of a failed development gate

Giovanni Del Papa
3 links
Zenodo (CERN European Organization for Nuclear Research)
Domain Adaptation and Few-Shot Learning
preprint

chem v5: a QLoRA chemistry adapter for Qwen3-8B. Sealed-test evidence of a +5.0-point gain on multiple-choice chemistry, and an honest account of a failed development gate

Giovanni Del Papa
preprint en

Abstract

chem v5 is a QLoRA adapter (rank 16, about 43.6 M trainable parameters) for Qwen3-8B (Apache-2.0), trained for one epoch on 11,839 permissively licensed chemistry examples on a single 12 GB laptop GPU. Base and adapter were compared zero-shot, non-thinking, with letter-probability scoring and 4-bit NF4 inference on three held-out multiple-choice chemistry benchmarks, with paired bootstrap 95% confidence intervals (10,000 resamples). Sealed test (2,529 questions, run once on 17 November 2026): pooled accuracy 47.61% → 52.63% (+5.02 points, 95% CI +3.24 to +6.76); MMLU chemistry 63.95 → 67.81 (+3.86, CI −0.43 to +8.15), ChemBench MCQ 54.82 → 58.22 (+3.39, CI +1.13 to +5.72), MMLU-Pro chemistry 29.13 → 37.58 (+8.45, CI +5.17 to +11.73); 328 items gained, 201 lost; pre-specified gate passed. Development split (1,083 questions), same rule: pooled +1.20 points (CI −1.29 to +3.79), ChemBench −1.40; gate failed. Both results are reported side by side, together with a crash and partial re-run during the sealed test and a discussion of the dev/test gap. Italian and English versions are included. Author: Giovanni Del Papa, Independent researcher, Italy. chem v5 è un adattatore QLoRA (rango 16, circa 43,6 M parametri addestrabili) per Qwen3-8B (Apache-2.0), addestrato per un'epoca su 11.839 esempi di chimica con licenze permissive su una sola GPU da portatile da 12 GB. Test sigillato (2.529 domande, 17 novembre 2026): accuratezza complessiva 47,61% → 52,63% (+5,02 punti, IC 95% da +3,24 a +6,76), gate superato. Split di sviluppo (1.083 domande): +1,20 punti (IC da −1,29 a +3,79), gate non superato. Entrambi i risultati sono riportati affiancati. Version 1.1 removes a leftover draft line under the title. There are no content changes.

Zenodo (CERN European Organization for Nuclear Research)
Domain Adaptation and Few-Shot Learning
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon