Plausible citations, unsupported claims, and nationality-dependent recommendations: hidden failure modes of large language models in gynecologic oncology
Large language models increasingly answer gynecologic-oncology questions with citations and management recommendations. In a preregistered 308-item benchmark (252 confirmatory) administered to GPT-5.5, Claude Opus 4.8, and Gemini 3.1 Pro, we tested whether cited papers support the claims they accompany, whether unsupported citations track the volume of indexed literature, and whether otherwise-invariant recommendations change with patient nationality. Citations were verified for existence, record identity, and claim support by an automated judge validated against two blinded gynecologic oncologists and a second judge model. All three models answered ancestry-specific questions correctly (24 of 24). Strict fabrication (non-existent or misattributed citations) was unrelated to literature coverage (β = −0.19, p = 0.22), whereas citation of real, on-topic papers lacking the claimed fact rose steeply as coverage fell (β = −1.12, p = 2 × 10⁻⁸; failure rates 0.67, 0.60, and 0.26 across tertiles), with a similar gradient in the 48 exploration-only items. Nationality-dependent violations of invariant management reached 0.11–0.67 across models when the care setting was unstated versus 0.00–0.13 when a fully resourced setting was stated. Legitimate population differences were preserved. Evaluations should score claim-level citation support and specify the care setting; clinicians should verify that cited sources support the specific claim.
Authors
- Byoung‐Gie Kim (ORCID: https://orcid.org/0000-0002-0572-8450)
- Joseph J. Noh (ORCID: https://orcid.org/0000-0002-3132-8709)
- Jeong-Sig Kim
- Se Ik Kim
- Jae Hyun Cho
- Junhwan Kim
Institutions
- Seoul National University (KR)
- Soonchunhyang University Hospital Seoul (KR)
- Seoul National University Bundang Hospital (KR)
- Chung-Ang University Gwangmyeong Hospital (KR)
Publication Details
- Journal
- Research Square
- Published
- 2026-11-19
- DOI
- https://doi.org/10.21203/rs.3.rs-11146954/v1
- Primary Topic
- Artificial Intelligence in Healthcare and Education
- Type
- preprint