Psychotherapy remission prediction models show limited cross-clinic generalizability in routine outpatient care

Abstract Clinical prediction models for psychotherapy outcomes have the potential to assist therapists in planning a patient’s individual treatment, but their clinical utility depends on maintaining predictive performance across settings. We assessed prediction of post-treatment remission in 2,002 patients with internalizing disorders treated across eight German university outpatient clinics. Penalized logistic-regression models were evaluated within clinics, in pooled clinic data, and using leave-one-clinic-out validation. Within clinics, mean AUC was 0.67, balanced accuracy 0.62, and Brier score 0.21. In leave-one-clinic-out validation, mean AUC and balanced accuracy declined from 0.60 and 0.57 in corresponding held-out test samples to 0.53 and 0.52 in held-out clinics, respectively. The mean Brier score increased from 0.23 to 0.25, but this difference was not statistically significant (p = 0.18). Cross-clinic performance loss was therefore clearest for discrimination and threshold-dependent classification, while overall probabilistic prediction error showed a smaller change. These findings highlight the broader challenge of transporting psychotherapy outcome predictions across heterogeneous clinical settings.

Authors

Institutions

Publication Details

Journal
Scientific Reports
Published
2026-09-19
DOI
https://doi.org/10.1038/s41598-026-70047-x
Primary Topic
Digital Mental Health Interventions
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
article

Psychotherapy remission prediction models show limited cross-clinic generalizability in routine outpatient care

Anne‐Kathrin Bräscher, Sylvia Helbig‐Lang, Stephan Bartholdy, Tobias Teismann et al.
Scientific Reports
Digital Mental Health Interventions
article

Psychotherapy remission prediction models show limited cross-clinic generalizability in routine outpatient care

Anne‐Kathrin Bräscher, Sylvia Helbig‐Lang, Stephan Bartholdy, Tobias Teismann, Monika Equit, Ulrike Lueken, Patrizia Odyniec, Julia Velten, Almut Rudolph, Gabriele Wilz, Anya Pedersen, Jürgen Margraf, Katja Werheid, Tim Klucken, Henning Schöttke, Andrea Hermann, Thomas Forkmann, Hanna Christiansen, Elisa Berger, Jürgen Hoyer, Lydia Fehm, Ulrike Willutzki, Lars Schulze, Georg W. Alpers, Anke Kirsch, Tania Lincoln, Wolfgang Lutz, Rudolf Stark, Tina In-Albon, Peter Weller, Timo Brockmeyer, Kevin Hilbert, Anne-Katrin Risch, Eva-Lotta Brakemeier, Julia Anna Glombiewski, Julian Rubel, Brian Schwartz
article en

Abstract

Abstract Clinical prediction models for psychotherapy outcomes have the potential to assist therapists in planning a patient’s individual treatment, but their clinical utility depends on maintaining predictive performance across settings. We assessed prediction of post-treatment remission in 2,002 patients with internalizing disorders treated across eight German university outpatient clinics. Penalized logistic-regression models were evaluated within clinics, in pooled clinic data, and using leave-one-clinic-out validation. Within clinics, mean AUC was 0.67, balanced accuracy 0.62, and Brier score 0.21. In leave-one-clinic-out validation, mean AUC and balanced accuracy declined from 0.60 and 0.57 in corresponding held-out test samples to 0.53 and 0.52 in held-out clinics, respectively. The mean Brier score increased from 0.23 to 0.25, but this difference was not statistically significant (p = 0.18). Cross-clinic performance loss was therefore clearest for discrimination and threshold-dependent classification, while overall probabilistic prediction error showed a smaller change. These findings highlight the broader challenge of transporting psychotherapy outcome predictions across heterogeneous clinical settings.

Scientific ReportsVol. 16(1)
Trier University of Applied Sciences (DE), University of Kaiserslautern (DE), Witten/Herdecke University (DE), Universität Hamburg (DE), Philipps University of Marburg (DE), Osnabrück University (DE), University of Mannheim (DE), Johannes Gutenberg University Mainz (DE), Justus-Liebig-Universität Gießen (DE), Bielefeld University (DE), University of Erfurt (DE), University of Siegen (DE), University of Münster (DE), Christian-Albrechts-Universität zu Kiel (DE), Universität Greifswald (DE), Humboldt-Universität zu Berlin (DE), University Medical Center of the Johannes Gutenberg University Mainz (DE), Nephrologisches Zentrum Goettingen (DE), University Hospital Leipzig (DE), GFZ Helmholtz Centre for Geosciences (DE), University of Duisburg-Essen (DE), FH Münster (DE), University of Applied Sciences Kaiserslautern (DE), Freie Universität Berlin (DE), Friedrich Schiller University Jena (DE), Technische Universität Dresden (DE), Universität Trier (DE), Saarland University (DE), Leipzig University (DE)
Reduced inequalities
Openalex Percentile: Top 9%
Digital Mental Health Interventions
DOIOpen Access PDFOpenAlex
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.