Prompt-dependent readability and accuracy of Generative Pre-trained Transformer-5 thinking responses to breast imaging questions derived from European Society of Breast Imaging patient recommendations

Prompt-dependent readability and accuracy of Generative Pre-trained Transformer-5 thinking responses to breast imaging questions derived from European Society of Breast Imaging patient recommendations PURPOSEAlthough professional societies provide standardized resources, the reliability of online health content remains inconsistent, particularly as large language models (LLMs) become a growing source of patient information.Therefore, the primary objective of this study was to determine whether patient-focused prompt optimization improves the readability of Generative Pre-trained Transformer-5 (GPT-5) Thinking responses compared with standard prompting while evaluating potential changes in expert-rated factual accuracy using educational materials published by the European Society of Breast Imaging (EUSOBI). METHODSAll GPT-5 Thinking outputs were prospectively generated in January 2026.The dataset included frequently asked questions (FAQs) and special information notes (SINs) derived from EUSOBI articles, each submitted under two distinct prompts (standard-Prompt 1 vs. patient-focused-Prompt 2).Readability was quantified using four indices with an averaged composite "reading grade level." Accuracy was scored by two radiologists using a 5-point Likert scale. RESULTSA total of 48 questions (15 FAQs and 33 SIN-derived questions) were each answered under two prompt conditions, yielding 96 responses for analysis.Prompt 2 produced significantly lower readability levels than Prompt 1 across all indices (P < 0.001).These improvements occurred without significant differences in word count (P = 0.12) or sentence count (P = 0.63).The mean accuracy scores remained high and comparable between prompts for all questions (4.54 ± 0.63 vs. 4.52 ± 0.59, P = 0.75).The inter-rater agreement was almost perfect for both the FAQ and SIN-derived question sets (κ = 0.82 and 0.81, respectively). CONCLUSIONPatient-focused prompting substantially reduced the linguistic complexity and reading grade level of GPT-5 Thinking responses without a statistically significant reduction in expert-rated accuracy, demonstrating its ability to produce more accessible breast imaging content, although the resulting grade level remained above widely recommended patient-education thresholds. CLINICAL SIGNIFICANCEA simple, low-cost prompt modification can substantially improve the readability of LLM-generated patient education content in breast imaging without a measurable reduction in expert-rated accuracy, although further work is needed to confirm this translates into improved patient comprehension.

Authors

Institutions

Publication Details

Journal
Diagnostic and Interventional Radiology
Published
2026-11-12
DOI
https://doi.org/10.4274/dir.2026.264309
Primary Topic
Artificial Intelligence in Healthcare and Education
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Prompt-dependent readability and accuracy of Generative Pre-trained Transformer-5 thinking responses to breast imaging questions derived from European Society of Breast Imaging patient recommendations

Çağlar Uzun, Sena Bozer Uludağ, Zeynep Eskalen, Gökhan Uludağ et al.
Diagnostic and Interventional Radiology
Artificial Intelligence in Healthcare and Education
article

Prompt-dependent readability and accuracy of Generative Pre-trained Transformer-5 thinking responses to breast imaging questions derived from European Society of Breast Imaging patient recommendations

Çağlar Uzun, Sena Bozer Uludağ, Zeynep Eskalen, Gökhan Uludağ, Ebru Düşünceli Atman
article en

Abstract

Prompt-dependent readability and accuracy of Generative Pre-trained Transformer-5 thinking responses to breast imaging questions derived from European Society of Breast Imaging patient recommendations PURPOSEAlthough professional societies provide standardized resources, the reliability of online health content remains inconsistent, particularly as large language models (LLMs) become a growing source of patient information.Therefore, the primary objective of this study was to determine whether patient-focused prompt optimization improves the readability of Generative Pre-trained Transformer-5 (GPT-5) Thinking responses compared with standard prompting while evaluating potential changes in expert-rated factual accuracy using educational materials published by the European Society of Breast Imaging (EUSOBI). METHODSAll GPT-5 Thinking outputs were prospectively generated in January 2026.The dataset included frequently asked questions (FAQs) and special information notes (SINs) derived from EUSOBI articles, each submitted under two distinct prompts (standard-Prompt 1 vs. patient-focused-Prompt 2).Readability was quantified using four indices with an averaged composite "reading grade level." Accuracy was scored by two radiologists using a 5-point Likert scale. RESULTSA total of 48 questions (15 FAQs and 33 SIN-derived questions) were each answered under two prompt conditions, yielding 96 responses for analysis.Prompt 2 produced significantly lower readability levels than Prompt 1 across all indices (P < 0.001).These improvements occurred without significant differences in word count (P = 0.12) or sentence count (P = 0.63).The mean accuracy scores remained high and comparable between prompts for all questions (4.54 ± 0.63 vs. 4.52 ± 0.59, P = 0.75).The inter-rater agreement was almost perfect for both the FAQ and SIN-derived question sets (κ = 0.82 and 0.81, respectively). CONCLUSIONPatient-focused prompting substantially reduced the linguistic complexity and reading grade level of GPT-5 Thinking responses without a statistically significant reduction in expert-rated accuracy, demonstrating its ability to produce more accessible breast imaging content, although the resulting grade level remained above widely recommended patient-education thresholds. CLINICAL SIGNIFICANCEA simple, low-cost prompt modification can substantially improve the readability of LLM-generated patient education content in breast imaging without a measurable reduction in expert-rated accuracy, although further work is needed to confirm this translates into improved patient comprehension.

Diagnostic and Interventional Radiology
Ankara University (TR), Sağlık Bilimleri Üniversitesi (TR), Ankara Bilkent City Hospital (TR)
Good health and well-being
Artificial Intelligence in Healthcare and Education
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon