Preterm Small-for-Gestational-Age Risk Estimates Using Large Language Models Versus the FMF Competing-Risks Model

Introduction: The midgestation prediction of preterm small-for-gestational-age (SGA) birth could use the externally validated Fetal Medicine Foundation (FMF) competing-risks model, integrating maternal characteristics and obstetric history with fetal biometry and biophysical markers. Large language model (LLM) systems are increasingly used for pregnancy-related health information, but how their numerical risk estimates respond to clinical inputs compared with this model is not known. Objective: To compare the direction and relative magnitude of changes in numerical estimates of preterm SGA risk from eight LLMs and the FMF competing-risks model when individual predictors are varied, and examine how agreement relates to the calculation methods reported by the LLMs. Methods: We generated 108 synthetic singleton pregnancy scenarios by varying one of 16 clinical variables (19 encoded predictors) at a time around low-risk multiparous and nulliparous profiles assessed at 22 weeks. Each LLM was queried once through its consumer interface for the probability of birthweight below the 10th percentile with delivery before 37 weeks; temperature and tool use could not be controlled. FMF probabilities were calculated from the published equations. Local interpretable model-agnostic explanation (LIME)-inspired weighted linear surrogates were fitted to logit-transformed probabilities and compared with FMF using an exploratory composite alignment score, combining directional concordance, Spearman correlation, cosine similarity, and normalized root mean squared error similarity. Absolute agreement was assessed on the logit scale. Results: The largest FMF surrogate coefficients were chronic hypertension, diabetes mellitus, and smoking among binary predictors, and uterine artery pulsatility index, estimated fetal weight Z-score, and gestational age at the previous delivery among continuous predictors. Composite alignment with FMF was highest for GPT-6 (0.999), Mistral (0.988), and Grok (0.917), followed by Claude (0.894), DeepSeek (0.769), Meta (0.752), Qwen (0.600), and Copilot (0.492). Diabetes mellitus received a negative surrogate coefficient in four LLMs, although only DeepSeek returned a lower probability with diabetes than without. Predicted probabilities agreed closely with FMF only for GPT-6; the other LLMs predicted higher risks, with odds on average 1.2 to 9 times those of FMF. GPT-6, Mistral, and Grok reported implementing the published FMF algorithm, although this cannot be verified; Claude, DeepSeek, and Qwen described approximate logistic models; and Copilot and Meta displayed no process messages. Conclusions: Under a single prompt and uncontrolled consumer access conditions, similarity to the FMF model varied widely and was highest for LLMs reporting use of the published algorithm, while predicted risks often diverged, even when the effects of individual risk factors were similar. Verification of the calculations and validation against observed pregnancy outcomes are required before clinical application.

Authors

Institutions

Publication Details

Journal
Diagnostics
Published
2026-11-12
DOI
https://doi.org/10.3390/diagnostics16223742
Primary Topic
Artificial Intelligence in Healthcare and Education
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Preterm Small-for-Gestational-Age Risk Estimates Using Large Language Models Versus the FMF Competing-Risks Model

Alexandra Ursache, Dragoş Nemescu, Emil Anton, Denisa Oana Zelinschi et al.
Diagnostics
Artificial Intelligence in Healthcare and Education
article

Preterm Small-for-Gestational-Age Risk Estimates Using Large Language Models Versus the FMF Competing-Risks Model

Alexandra Ursache, Dragoş Nemescu, Emil Anton, Denisa Oana Zelinschi, Eduard Cristian Mihoci, Oriana-Maria Onicescu, Sorana Anton
article en

Abstract

Introduction: The midgestation prediction of preterm small-for-gestational-age (SGA) birth could use the externally validated Fetal Medicine Foundation (FMF) competing-risks model, integrating maternal characteristics and obstetric history with fetal biometry and biophysical markers. Large language model (LLM) systems are increasingly used for pregnancy-related health information, but how their numerical risk estimates respond to clinical inputs compared with this model is not known. Objective: To compare the direction and relative magnitude of changes in numerical estimates of preterm SGA risk from eight LLMs and the FMF competing-risks model when individual predictors are varied, and examine how agreement relates to the calculation methods reported by the LLMs. Methods: We generated 108 synthetic singleton pregnancy scenarios by varying one of 16 clinical variables (19 encoded predictors) at a time around low-risk multiparous and nulliparous profiles assessed at 22 weeks. Each LLM was queried once through its consumer interface for the probability of birthweight below the 10th percentile with delivery before 37 weeks; temperature and tool use could not be controlled. FMF probabilities were calculated from the published equations. Local interpretable model-agnostic explanation (LIME)-inspired weighted linear surrogates were fitted to logit-transformed probabilities and compared with FMF using an exploratory composite alignment score, combining directional concordance, Spearman correlation, cosine similarity, and normalized root mean squared error similarity. Absolute agreement was assessed on the logit scale. Results: The largest FMF surrogate coefficients were chronic hypertension, diabetes mellitus, and smoking among binary predictors, and uterine artery pulsatility index, estimated fetal weight Z-score, and gestational age at the previous delivery among continuous predictors. Composite alignment with FMF was highest for GPT-6 (0.999), Mistral (0.988), and Grok (0.917), followed by Claude (0.894), DeepSeek (0.769), Meta (0.752), Qwen (0.600), and Copilot (0.492). Diabetes mellitus received a negative surrogate coefficient in four LLMs, although only DeepSeek returned a lower probability with diabetes than without. Predicted probabilities agreed closely with FMF only for GPT-6; the other LLMs predicted higher risks, with odds on average 1.2 to 9 times those of FMF. GPT-6, Mistral, and Grok reported implementing the published FMF algorithm, although this cannot be verified; Claude, DeepSeek, and Qwen described approximate logistic models; and Copilot and Meta displayed no process messages. Conclusions: Under a single prompt and uncontrolled consumer access conditions, similarity to the FMF model varied widely and was highest for LLMs reporting use of the published algorithm, while predicted risks often diverged, even when the effects of individual risk factors were similar. Verification of the calculations and validation against observed pregnancy outcomes are required before clinical application.

DiagnosticsVol. 16(22)
Alexandru Ioan Cuza University (RO), Grigore T. Popa University of Medicine and Pharmacy (RO)
Good health and well-being
Artificial Intelligence in Healthcare and Education
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon