Selective retrieval-augmented answering for medical multiple-choice QA with MedGemma-27B: results-only report (single-use public test) / RAG selettivo per QA medico a scelta multipla con MedGemma-27B: rapporto solo-risultati

English. A confidence- and relevance-gated retrieval layer on top of MedGemma-27B (bf16, vLLM, greedy) answers with retrieved passages only for questions on which the base model is uncertain and a Qwen3-4B reranker finds a highly relevant passage (tc = 0.8, τ = 0.93245, k = 3; thresholds calibrated on train/validation only and frozen before testing). With a retrieval corpus that includes non-commercially licensed sources ("full" variant), accuracy on a single-use public test of n = 6,545 medical multiple-choice items rises from 70.15 % to 71.00 % (Δ = +0.86 pp, 95 % bootstrap CI [+0.55, +1.16], exact McNemar p = 4.5·10-8; 81 items fixed, 25 broken; 245 items gated). The same pipeline restricted to commercially licensed sources ("comm") does not beat the base (+0.08 pp, CI [−0.41, +0.55]) and is therefore switched off. Because the effective corpus is not commercially licensable, only these aggregate results are published: no index, passages or source data. The report includes the formulas, figures, per-benchmark tables with CIs and tests, and the funnel of the gating stages. Cost of the run: about USD 0.32 (A100 80 GB, 0.227 h). Hashes: frozen.json sha256 ee56a54b52b0fa178dca0f78d7665cfb158a7eeace8de171aec7100c5622ab99; artifacts.tgz sha256 8fb79b571d47fcc5f0c9bb8ed0225eb76153cdaa0bfa45bd9975180560f1e4b3. Not for clinical use. Italiano. Uno strato RAG controllato da soglie di confidenza e rilevanza sopra MedGemma-27B risponde con i documenti solo sulle domande incerte per cui un reranker trova un passaggio molto rilevante. Con un corpus che include fonti non commerciali (variante «completa»), su un test pubblico a uso singolo di n = 6.545 item l’accuratezza passa dal 70,15 % al 71,00 % (Δ = +0,86 pp, IC 95 % [+0,55; +1,16], McNemar esatto p = 4,5·10-8). La variante limitata a fonti commerciali non supera la base (+0,08 pp) ed è spenta. Poiché il corpus efficace non è commercializzabile, si pubblicano solo questi risultati aggregati: nessun indice, passaggio o dato. Files: article_en.pdf, article_it.pdf, code.zip (statistics, figures, LaTeX sources, aggregate results). Licences: article and figures CC BY 4.0; code Apache-2.0.

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-11-02
DOI
https://doi.org/10.5281/zenodo.23895101
Primary Topic
Information Retrieval and Search Behavior
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
preprint

Selective retrieval-augmented answering for medical multiple-choice QA with MedGemma-27B: results-only report (single-use public test) / RAG selettivo per QA medico a scelta multipla con MedGemma-27B: rapporto solo-risultati

Giovanni Del Papa
4 links
Zenodo (CERN European Organization for Nuclear Research)
Information Retrieval and Search Behavior
preprint

Selective retrieval-augmented answering for medical multiple-choice QA with MedGemma-27B: results-only report (single-use public test) / RAG selettivo per QA medico a scelta multipla con MedGemma-27B: rapporto solo-risultati

Giovanni Del Papa
preprint en

Abstract

English. A confidence- and relevance-gated retrieval layer on top of MedGemma-27B (bf16, vLLM, greedy) answers with retrieved passages only for questions on which the base model is uncertain and a Qwen3-4B reranker finds a highly relevant passage (tc = 0.8, τ = 0.93245, k = 3; thresholds calibrated on train/validation only and frozen before testing). With a retrieval corpus that includes non-commercially licensed sources ("full" variant), accuracy on a single-use public test of n = 6,545 medical multiple-choice items rises from 70.15 % to 71.00 % (Δ = +0.86 pp, 95 % bootstrap CI [+0.55, +1.16], exact McNemar p = 4.5·10-8; 81 items fixed, 25 broken; 245 items gated). The same pipeline restricted to commercially licensed sources ("comm") does not beat the base (+0.08 pp, CI [−0.41, +0.55]) and is therefore switched off. Because the effective corpus is not commercially licensable, only these aggregate results are published: no index, passages or source data. The report includes the formulas, figures, per-benchmark tables with CIs and tests, and the funnel of the gating stages. Cost of the run: about USD 0.32 (A100 80 GB, 0.227 h). Hashes: frozen.json sha256 ee56a54b52b0fa178dca0f78d7665cfb158a7eeace8de171aec7100c5622ab99; artifacts.tgz sha256 8fb79b571d47fcc5f0c9bb8ed0225eb76153cdaa0bfa45bd9975180560f1e4b3. Not for clinical use. Italiano. Uno strato RAG controllato da soglie di confidenza e rilevanza sopra MedGemma-27B risponde con i documenti solo sulle domande incerte per cui un reranker trova un passaggio molto rilevante. Con un corpus che include fonti non commerciali (variante «completa»), su un test pubblico a uso singolo di n = 6.545 item l’accuratezza passa dal 70,15 % al 71,00 % (Δ = +0,86 pp, IC 95 % [+0,55; +1,16], McNemar esatto p = 4,5·10-8). La variante limitata a fonti commerciali non supera la base (+0,08 pp) ed è spenta. Poiché il corpus efficace non è commercializzabile, si pubblicano solo questi risultati aggregati: nessun indice, passaggio o dato. Files: article_en.pdf, article_it.pdf, code.zip (statistics, figures, LaTeX sources, aggregate results). Licences: article and figures CC BY 4.0; code Apache-2.0.

Zenodo (CERN European Organization for Nuclear Research)
Information Retrieval and Search Behavior
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon
Selective retrieval-augmented answering for medical multiple-choice QA with MedGemma-27B: results-only report (single-use public test) / RAG selettivo per QA medico a scelta multipla con MedGemma-27B: rapporto solo-risultati — Giovanni Del Papa · Zenodo (CERN European Organization for Nuclear Research) (2026) | TGRS Research Map | TGRS