Temporal stability of an electronic health record-driven machine learning model predicting type 2 diabetes among patients with mental illness

OBJECTIVES: We previously trained and tested a machine learning model to predict incident type 2 diabetes (T2D) in patients with mental illness using electronic health record (EHR) data from routine clinical practice. This hyperparameter-tuned XGBoost model - based on information on sex, age and mean HbA1c within the past two years - predicts whether a patient will meet criteria for T2D in the two years following each contact to psychiatric services. The performance of such a model may drift over time due to changes in standards of care, baseline risk, registration procedures, etc. Therefore, we set out to examine its temporal stability. METHODS: In the present study, we fitted the model described above on EHR data from January 1, 2013 to December 31, 2017, and examined its stability in predicting T2D two years into the future for patients with contacts in 2018, 2019, 2020, 2021, 2022, and 2023, respectively. Performance was quantified using the area under the receiver operating characteristic curve (AUROC). RESULTS: The model showed good discriminative performance across all years (AUROC for 2018 = 0.84, 2019 = 0.83, 2020 = 0.87, 2021 = 0.80, 2022 = 0.84, 2023 = 0.80), with a linear model estimating a decrease in AUROC for T2D prediction of only 0.004 per year (95%CI: -0.024; 0.016). Secondary analyses suggested that models trained on recent data (past year) may achieve similar or better discrimination than models trained on longer historical periods. CONCLUSIONS: The temporal stability of the T2D prediction model supports its potential for clinical implementation, facilitating early detection and tailored interventions, while highlighting the importance of continued evaluation.

Authors

Institutions

Publication Details

Journal
Acta Neuropsychiatrica
Published
2026-11-13
DOI
https://doi.org/10.1017/neu.2026.10105
Primary Topic
Machine Learning in Healthcare
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Temporal stability of an electronic health record-driven machine learning model predicting type 2 diabetes among patients with mental illness

Søren Dinesen Østergaard, Andreas Aalkjær Danielsen, Frida Hæstrup
3 links
Acta Neuropsychiatrica
Machine Learning in Healthcare
article

Temporal stability of an electronic health record-driven machine learning model predicting type 2 diabetes among patients with mental illness

Søren Dinesen Østergaard, Andreas Aalkjær Danielsen, Frida Hæstrup
article en

Abstract

OBJECTIVES: We previously trained and tested a machine learning model to predict incident type 2 diabetes (T2D) in patients with mental illness using electronic health record (EHR) data from routine clinical practice. This hyperparameter-tuned XGBoost model - based on information on sex, age and mean HbA1c within the past two years - predicts whether a patient will meet criteria for T2D in the two years following each contact to psychiatric services. The performance of such a model may drift over time due to changes in standards of care, baseline risk, registration procedures, etc. Therefore, we set out to examine its temporal stability. METHODS: In the present study, we fitted the model described above on EHR data from January 1, 2013 to December 31, 2017, and examined its stability in predicting T2D two years into the future for patients with contacts in 2018, 2019, 2020, 2021, 2022, and 2023, respectively. Performance was quantified using the area under the receiver operating characteristic curve (AUROC). RESULTS: The model showed good discriminative performance across all years (AUROC for 2018 = 0.84, 2019 = 0.83, 2020 = 0.87, 2021 = 0.80, 2022 = 0.84, 2023 = 0.80), with a linear model estimating a decrease in AUROC for T2D prediction of only 0.004 per year (95%CI: -0.024; 0.016). Secondary analyses suggested that models trained on recent data (past year) may achieve similar or better discrimination than models trained on longer historical periods. CONCLUSIONS: The temporal stability of the T2D prediction model supports its potential for clinical implementation, facilitating early detection and tailored interventions, while highlighting the importance of continued evaluation.

Acta NeuropsychiatricaVol. 38
Deakin University (AU), Aarhus University (DK), Aarhus University Hospital (DK)
Good health and well-being
Machine Learning in Healthcare
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon