Generalizability of clinical prediction models in mental health

Concerns about the generalizability of machine learning models in mental health arise, partly due to sampling effects and data disparities between research cohorts and real-world populations. We aimed to investigate whether a machine learning model trained solely on easily accessible and low-cost clinical data can predict depressive symptom severity in unseen, independent datasets from various research and real-world clinical contexts. This observational multi-cohort study included 3021 participants (62.03% females, M<sub>Age</sub> = 36.27 years, range 15-81) from ten European research and clinical settings, all diagnosed with an affective disorder. We firstly compared research and real-world inpatients from the same treatment center using 76 clinical and sociodemographic variables. An elastic net algorithm with ten-fold cross-validation was then applied to develop a sparse machine learning model for predicting depression severity based on the top five features (global functioning, extraversion, neuroticism, emotional abuse in childhood, and somatization). Model generalizability was tested across nine external samples. The model reliably predicted depression severity across all samples (r = 0.60, SD = 0.089, p < 0.0001) and in each individual external sample, ranging in performance from r = 0.48 in a real-world general population sample to r = 0.73 in real-world inpatients. These results suggest that machine learning models trained on sparse clinical data have the potential to predict illness severity across diverse settings, offering insights that could inform the development of more generalizable tools for use in routine psychiatric data analysis.

Authors

Institutions

Publication Details

Journal
Molecular Psychiatry
Published
2025-03-19
DOI
https://doi.org/10.1038/s41380-025-02950-0
Citations
15
Primary Topic
Mental Health Research Topics
Type
article
Field-Weighted Citation Impact
31.77

Funders

Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

Generalizability of clinical prediction models in mental health

Ronny Redlich, Alessandro Bertolino, Andreas Jansen, Ulrike Buhlmann et al.
15 citations
Molecular Psychiatry
Mental Health Research Topics
31.77
article

Generalizability of clinical prediction models in mental health

Ronny Redlich, Alessandro Bertolino, Andreas Jansen, Ulrike Buhlmann, Benjamín Straube, Lena Florentine Köhler, Franco Fabbro, Janik Goltermann, Rogério Blitz, Susanne L. Meinert, Esther Zwiky, M. Rau, Sharmili Edwin Thanarajah, Matthias A. Reinhard, Katharina Thiel, Ramona Leenings, Nils R. Winter, Eva Meisenzahl‐Lechner, Maike Richter, Florian Thomas‐Odenthal, Nina Alexander, Daniel Emden, Tim Hahn, Rachel Upthegrove, Igor Nenadić, Nils Opel, Frederike Stein, Paolo Brambilla, Paula Usemann, Udo Dannlowski, Tilo T. J. Kircher, Nexhmedin Morina, Frank Padberg, Janka Massag, Anna Kraus, Katharina Brosch, Rafael Mikolajczyk, Stefan Borgwardt, Mitja D. Back, Joseph Kambeitz, Lea Teutenberg, Nikolaos Koutsouleris, Rebekka Lencer, Janette Ratzsch, Navid Schürmeyer, Renata Falguera, Hamidreza Jamalabadi, Tiana Borgers, Kira Flinkenflügel, Raimo K. R. Salonkangas
article en
15 citations

Abstract

Concerns about the generalizability of machine learning models in mental health arise, partly due to sampling effects and data disparities between research cohorts and real-world populations. We aimed to investigate whether a machine learning model trained solely on easily accessible and low-cost clinical data can predict depressive symptom severity in unseen, independent datasets from various research and real-world clinical contexts. This observational multi-cohort study included 3021 participants (62.03% females, M<sub>Age</sub> = 36.27 years, range 15-81) from ten European research and clinical settings, all diagnosed with an affective disorder. We firstly compared research and real-world inpatients from the same treatment center using 76 clinical and sociodemographic variables. An elastic net algorithm with ten-fold cross-validation was then applied to develop a sparse machine learning model for predicting depression severity based on the top five features (global functioning, extraversion, neuroticism, emotional abuse in childhood, and somatization). Model generalizability was tested across nine external samples. The model reliably predicted depression severity across all samples (r = 0.60, SD = 0.089, p < 0.0001) and in each individual external sample, ranging in performance from r = 0.48 in a real-world general population sample to r = 0.73 in real-world inpatients. These results suggest that machine learning models trained on sparse clinical data have the potential to predict illness severity across diverse settings, offering insights that could inform the development of more generalizable tools for use in routine psychiatric data analysis.

Molecular PsychiatryVol. 30(8)
Goethe University Frankfurt (DE), University of Udine (IT), Feinstein Institute for Medical Research (US), University of Turku (FI), Philipps University of Marburg (DE), University of Cologne (DE), King's College London (GB), University of Basel (CH), University of Milan (IT), Bielefeld University (DE), University of Münster (DE), LMU Klinikum (DE), Centre for Mental Health (GB), University of Oxford (GB), Max Planck Institute for Metabolism Research (DE), University Hospital Frankfurt (DE), Jena University Hospital (DE), Max Planck Institute of Psychiatry (DE), University Hospital Cologne (DE), Heinrich Heine University Düsseldorf (DE), University of Bari Aldo Moro (IT), Martin Luther University Halle-Wittenberg (DE), University of Birmingham (GB), Ludwig-Maximilians-Universität München (DE)
European Commission, Deutsche Forschungsgemeinschaft, Bundesministerium für Bildung und Forschung, Leibniz-Gemeinschaft
Good health and well-being
Openalex Percentile: Top 0%
Mental Health Research Topics
31.77
DOIOpen Access PDFOpenAlex
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.