Expert of Experts Verification and Alignment (EVAL) Framework for Large Language Models Safety in Gastroenterology

Large language models generate plausible text responses to medical questions, but inaccurate responses pose significant risks in medical decision-making. Grading LLM outputs to determine the best model or answer is time-consuming and impractical in clinical settings; therefore, we introduce EVAL (Expert-of-Experts Verification and Alignment) to streamline this process and enhance LLM safety for upper gastrointestinal bleeding (UGIB). We evaluated OpenAI's GPT-3.5/4/4o/o1-preview, Anthropic's Claude-3-Opus, Meta's LLaMA-2 (7B/13B/70B), and Mistral AI's Mixtral (7B) across 27 configurations, including zero-shot baseline, retrieval-augmented generation, and supervised fine-tuning. EVAL uses similarity-based ranking and a reward model trained on human-graded responses for rejection sampling. Among the employed similarity metrics, Fine-Tuned ColBERT achieved the highest alignment with human performance across three separate datasets (ρ = 0.81-0.91). The reward model replicated human grading with 87.9% of cases across temperature settings and significantly improved accuracy through rejection sampling by 8.36% overall. EVAL offers scalable potential to assess accuracy for high-stakes medical decision-making.

Authors

Institutions

Publication Details

Journal
npj Digital Medicine
Published
2025-05-03
DOI
https://doi.org/10.1038/s41746-025-01589-z
Citations
6
Primary Topic
Artificial Intelligence in Healthcare and Education
Type
article
Field-Weighted Citation Impact
3.93

Funders

Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

Expert of Experts Verification and Alignment (EVAL) Framework for Large Language Models Safety in Gastroenterology

Bradly C. Stadie, Loren Laine, Ian M. Gralnek, Kisung You et al.
6 citations
npj Digital Medicine
Artificial Intelligence in Healthcare and Education
3.93
article

Expert of Experts Verification and Alignment (EVAL) Framework for Large Language Models Safety in Gastroenterology

Bradly C. Stadie, Loren Laine, Ian M. Gralnek, Kisung You, Simone Kresevic, Guadalupe García–Tsao, Alan N. Barkun, Lory Saveria Crocè, Colleen Chan, Zengchang Pang, Joseph J.�Y. Sung, Jasjeet S. Sekhon, Ryan Chen, Sunny Chung, Theo Saarinen, Mauro Giuffrè, Dennis Shung, Miloš Ajčević, Youngmin Ko
article en
6 citations

Abstract

Large language models generate plausible text responses to medical questions, but inaccurate responses pose significant risks in medical decision-making. Grading LLM outputs to determine the best model or answer is time-consuming and impractical in clinical settings; therefore, we introduce EVAL (Expert-of-Experts Verification and Alignment) to streamline this process and enhance LLM safety for upper gastrointestinal bleeding (UGIB). We evaluated OpenAI's GPT-3.5/4/4o/o1-preview, Anthropic's Claude-3-Opus, Meta's LLaMA-2 (7B/13B/70B), and Mistral AI's Mixtral (7B) across 27 configurations, including zero-shot baseline, retrieval-augmented generation, and supervised fine-tuning. EVAL uses similarity-based ranking and a reward model trained on human-graded responses for rejection sampling. Among the employed similarity metrics, Fine-Tuned ColBERT achieved the highest alignment with human performance across three separate datasets (ρ = 0.81-0.91). The reward model replicated human grading with 87.9% of cases across temperature settings and significantly improved accuracy through rejection sampling by 8.36% overall. EVAL offers scalable potential to assess accuracy for high-stakes medical decision-making.

npj Digital MedicineVol. 8(1)
Northwestern University (US), Baruch College (US), University of Trieste (IT), Nanyang Technological University (SG), Technion – Israel Institute of Technology (IL), Yale University (US), Northwestern University (PH), Berry (United States) (US), McGill University (CA)
National Institutes of Health, National Institute of Diabetes and Digestive and Kidney Diseases
Openalex Percentile: Top 6%
Artificial Intelligence in Healthcare and Education
3.93
DOIOpen Access PDFOpenAlex
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.