Guidelines for Reporting Studies on Large Language Models in Radiology: An International Delphi Expert Survey

Large language models (LLMs) have transformative potential in radiology, including textual summaries, diagnostic decision support, proofreading, and image analysis. However, the rapid increase in studies investigating these models, along with the lack of standardized LLM-specific reporting practices, affects reproducibility, reliability, and clinical applicability. To address this, reporting guidelines for LLM studies in radiology were developed using a two-step process. First, a systematic review of LLM studies in radiology was conducted across PubMed, IEEE Xplore, and the ACM Digital Library, covering publications between May 2023 and March 2024. Of 511 screened studies, 57 were included to identify relevant aspects for the guidelines. Then, in a Delphi process, 20 international experts developed the final list of items for inclusion. Items consented as relevant were summarized into a structured checklist containing 32 items across six key categories: general information and data input; prompting and fine-tuning; performance metrics; ethics and data transparency; implementation, risks, and limitations; and further/optional aspects. The final FLAIR (Framework for LLM Assessment in Radiology) checklist aims to standardize reporting of LLM studies in radiology, fostering transparency, reproducibility, comparability, and clinical applicability to enhance clinical translation and patient care. © The Author(s) 2026. Published by the Radiological Society of North America under a CC BY 4.0 license. <i>Supplemental material is available for this article.</i>

Authors

Institutions

Publication Details

Journal
Radiology
Published
2026-02-01
DOI
https://doi.org/10.1148/radiol.250913
Primary Topic
Radiology practices and education
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

Guidelines for Reporting Studies on Large Language Models in Radiology: An International Delphi Expert Survey

Kai Roman Laukamp, T Persigehl, Honghan Wu, Tarik K. Alkasab et al.
Radiology
Radiology practices and education
article

Guidelines for Reporting Studies on Large Language Models in Radiology: An International Delphi Expert Survey

Kai Roman Laukamp, T Persigehl, Honghan Wu, Tarik K. Alkasab, Andra-Iza Iuga, Almut Zeeck, J Kleesiek, Simon Lennartz, Matthias Alexander Fink, Thomas Dratsch, Christoph Wald, Roman Johannes Gertz, Christian Blüthgen, Arash Bedayat, Felix Nensa, Wei Wang, Linda Moy, Benyou Wang, Nils Große Hokamp, Akshay Chaudhari, David Maintz, Jonathan Kottlors, Erik Ranschaert, Jakob Nikolas Kather, Keno Bressem, Tianming Liu, Pranav Rajpurkar
article en

Abstract

Large language models (LLMs) have transformative potential in radiology, including textual summaries, diagnostic decision support, proofreading, and image analysis. However, the rapid increase in studies investigating these models, along with the lack of standardized LLM-specific reporting practices, affects reproducibility, reliability, and clinical applicability. To address this, reporting guidelines for LLM studies in radiology were developed using a two-step process. First, a systematic review of LLM studies in radiology was conducted across PubMed, IEEE Xplore, and the ACM Digital Library, covering publications between May 2023 and March 2024. Of 511 screened studies, 57 were included to identify relevant aspects for the guidelines. Then, in a Delphi process, 20 international experts developed the final list of items for inclusion. Items consented as relevant were summarized into a structured checklist containing 32 items across six key categories: general information and data input; prompting and fine-tuning; performance metrics; ethics and data transparency; implementation, risks, and limitations; and further/optional aspects. The final FLAIR (Framework for LLM Assessment in Radiology) checklist aims to standardize reporting of LLM studies in radiology, fostering transparency, reproducibility, comparability, and clinical applicability to enhance clinical translation and patient care. © The Author(s) 2026. Published by the Radiological Society of North America under a CC BY 4.0 license. <i>Supplemental material is available for this article.</i>

RadiologyVol. 318(2)
Lahey Hospital and Medical Center (US), Harvard University (US), Sun Yat-sen University (CN), University of California, Los Angeles (US), University of Georgia (US), Heidelberg University (DE), University Medical Center Freiburg (DE), University Hospital Heidelberg (DE), TUM Klinikum (DE), Ghent University (BE), Massachusetts General Hospital (US), University Hospital of Zurich (CH), Lahey Medical Center (US), National Center for Tumor Diseases (DE), Chinese University of Hong Kong, Shenzhen (CN), Essen University Hospital (DE), Universitätsklinikum Aachen (DE), The First Affiliated Hospital, Sun Yat-sen University (CN), Farr Institute (GB), Deutsches Herzzentrum München (DE), University Hospital Cologne (DE), Institute for Frontier Areas of Psychology and Mental Health (DE), University Hospital Carl Gustav Carus (DE), Artificial Intelligence in Medicine (Canada) (CA), New York University (US), University of Glasgow (GB), Stanford University (US)
Gender equality
Openalex Percentile: Top 69%
Radiology practices and education
DOIOpen Access PDFOpenAlex
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.