Preterm Small-for-Gestational-Age Risk Estimates Using Large Language Models Versus the FMF Competing-Risks Model
Introduction: The midgestation prediction of preterm small-for-gestational-age (SGA) birth could use the externally validated Fetal Medicine Foundation (FMF) competing-risks model, integrating maternal characteristics and obstetric history with fetal biometry and biophysical markers. Large language model (LLM) systems are increasingly used for pregnancy-related health information, but how their numerical risk estimates respond to clinical inputs compared with this model is not known. Objective: To compare the direction and relative magnitude of changes in numerical estimates of preterm SGA risk from eight LLMs and the FMF competing-risks model when individual predictors are varied, and examine how agreement relates to the calculation methods reported by the LLMs. Methods: We generated 108 synthetic singleton pregnancy scenarios by varying one of 16 clinical variables (19 encoded predictors) at a time around low-risk multiparous and nulliparous profiles assessed at 22 weeks. Each LLM was queried once through its consumer interface for the probability of birthweight below the 10th percentile with delivery before 37 weeks; temperature and tool use could not be controlled. FMF probabilities were calculated from the published equations. Local interpretable model-agnostic explanation (LIME)-inspired weighted linear surrogates were fitted to logit-transformed probabilities and compared with FMF using an exploratory composite alignment score, combining directional concordance, Spearman correlation, cosine similarity, and normalized root mean squared error similarity. Absolute agreement was assessed on the logit scale. Results: The largest FMF surrogate coefficients were chronic hypertension, diabetes mellitus, and smoking among binary predictors, and uterine artery pulsatility index, estimated fetal weight Z-score, and gestational age at the previous delivery among continuous predictors. Composite alignment with FMF was highest for GPT-6 (0.999), Mistral (0.988), and Grok (0.917), followed by Claude (0.894), DeepSeek (0.769), Meta (0.752), Qwen (0.600), and Copilot (0.492). Diabetes mellitus received a negative surrogate coefficient in four LLMs, although only DeepSeek returned a lower probability with diabetes than without. Predicted probabilities agreed closely with FMF only for GPT-6; the other LLMs predicted higher risks, with odds on average 1.2 to 9 times those of FMF. GPT-6, Mistral, and Grok reported implementing the published FMF algorithm, although this cannot be verified; Claude, DeepSeek, and Qwen described approximate logistic models; and Copilot and Meta displayed no process messages. Conclusions: Under a single prompt and uncontrolled consumer access conditions, similarity to the FMF model varied widely and was highest for LLMs reporting use of the published algorithm, while predicted risks often diverged, even when the effects of individual risk factors were similar. Verification of the calculations and validation against observed pregnancy outcomes are required before clinical application.
Authors
- Alexandra Ursache (ORCID: https://orcid.org/0000-0002-8207-7655)
- Dragoş Nemescu (ORCID: https://orcid.org/0000-0003-2345-1077)
- Emil Anton (ORCID: https://orcid.org/0000-0002-8244-0522)
- Denisa Oana Zelinschi
- Eduard Cristian Mihoci (ORCID: https://orcid.org/0009-0003-0791-0613)
- Oriana-Maria Onicescu (ORCID: https://orcid.org/0009-0002-4758-4309)
- Sorana Anton
Institutions
- Alexandru Ioan Cuza University (RO)
- Grigore T. Popa University of Medicine and Pharmacy (RO)
Publication Details
- Journal
- Diagnostics
- Published
- 2026-11-12
- DOI
- https://doi.org/10.3390/diagnostics16223742
- Primary Topic
- Artificial Intelligence in Healthcare and Education
- Type
- article
- Field-Weighted Citation Impact
- 0.00