Comparison of ChatGPT-Generated Patient Information on Testicular Cancer Across Two Assessment Periods: Clinical Accuracy, Response Quality, and Citation Support

Objective: The aim of this study was to compare clinical accuracy, response quality, and citation characteristics of ChatGPT-generated testicular cancer information across 2 assessments 20 months apart. Methods: Sixty-one patient-oriented questions derived from European Association of Urology (EAU) Patient Information were submitted to ChatGPT-4o in December 2024 and repeated unchanged using ChatGPT-5.6 Luna in August 2026. Two urologists independently assessed accuracy against contemporaneous EAU guidelines and rated quality using the Global Quality Score (GQS) and modified DISCERN (mDISCERN). Citation quantity, source quality, and source-to-claim support were evaluated. Only the first response was retained; repeated-generation variability was not assessed. Results: Fully accurate responses were 86.9% (53/61) in 2024 and 93.4% (57/61) in 2026 (P = .125). Of 8 clinically relevant inaccuracies in 2024, 4 were corrected, 3 improved but remained misleading, and 1 persisted. GQS increased from 3.75 ± 0.77 to 4.82 ± 0.41 and mDISCERN from 30.37 ± 4.65 to 36.80 ± 1.93 (both P < .001). Citations per response decreased from 5.97 ± 3.53 to 4.79 ± 1.19 (P = .042), while high-quality unique sources increased from 77.4% to 96.9% (P < .001). Fully supported sources were 80.1% and 86.4% (P = .050); 6 sources (2.6%) in 2026 did not support the claims. Conclusion: Response quality and the use of high-quality sources were greater in 2026, whereas changes in clinical accuracy and source-to-claim support were less clear. Clinically meaningful errors persisted in guideline-sensitive topics. Because different model versions and interfaces were assessed, the findings represent model- and interface-specific snapshots, not a controlled head-to-head comparison. Cite this article as: Ekenci BY, Altan M. Comparison of ChatGPT-generated patient information on testicular cancer across two assessment periods: Clinical accuracy, response quality, and citation support. Urol Res Pract. 2026;52, doi: 10.5152/ tud.2026.26067

Authors

Institutions

Publication Details

Journal
Urology Research and Practice
Published
2026-11-02
DOI
https://doi.org/10.5152/tud.2026.26067
Primary Topic
Artificial Intelligence in Healthcare and Education
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

Comparison of ChatGPT-Generated Patient Information on Testicular Cancer Across Two Assessment Periods: Clinical Accuracy, Response Quality, and Citation Support

Berk Yasin Ekenci, Mehmet Altan
4 links
Urology Research and Practice
Artificial Intelligence in Healthcare and Education
article

Comparison of ChatGPT-Generated Patient Information on Testicular Cancer Across Two Assessment Periods: Clinical Accuracy, Response Quality, and Citation Support

Berk Yasin Ekenci, Mehmet Altan
article en

Abstract

Objective: The aim of this study was to compare clinical accuracy, response quality, and citation characteristics of ChatGPT-generated testicular cancer information across 2 assessments 20 months apart. Methods: Sixty-one patient-oriented questions derived from European Association of Urology (EAU) Patient Information were submitted to ChatGPT-4o in December 2024 and repeated unchanged using ChatGPT-5.6 Luna in August 2026. Two urologists independently assessed accuracy against contemporaneous EAU guidelines and rated quality using the Global Quality Score (GQS) and modified DISCERN (mDISCERN). Citation quantity, source quality, and source-to-claim support were evaluated. Only the first response was retained; repeated-generation variability was not assessed. Results: Fully accurate responses were 86.9% (53/61) in 2024 and 93.4% (57/61) in 2026 (P = .125). Of 8 clinically relevant inaccuracies in 2024, 4 were corrected, 3 improved but remained misleading, and 1 persisted. GQS increased from 3.75 ± 0.77 to 4.82 ± 0.41 and mDISCERN from 30.37 ± 4.65 to 36.80 ± 1.93 (both P < .001). Citations per response decreased from 5.97 ± 3.53 to 4.79 ± 1.19 (P = .042), while high-quality unique sources increased from 77.4% to 96.9% (P < .001). Fully supported sources were 80.1% and 86.4% (P = .050); 6 sources (2.6%) in 2026 did not support the claims. Conclusion: Response quality and the use of high-quality sources were greater in 2026, whereas changes in clinical accuracy and source-to-claim support were less clear. Clinically meaningful errors persisted in guideline-sensitive topics. Because different model versions and interfaces were assessed, the findings represent model- and interface-specific snapshots, not a controlled head-to-head comparison. Cite this article as: Ekenci BY, Altan M. Comparison of ChatGPT-generated patient information on testicular cancer across two assessment periods: Clinical accuracy, response quality, and citation support. Urol Res Pract. 2026;52, doi: 10.5152/ tud.2026.26067

Urology Research and PracticeVol. 52
Ministry of Health (TR), Bayındır Hastanesi (TR)
Good health and well-being
Openalex Percentile: Top 22%
Artificial Intelligence in Healthcare and Education
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon