Multivariable prediction of functional outcome after first-episode psychosis: a crossover validation approach in EUFEST and PSYSCAN

Several multivariate prognostic models have been published to predict outcomes in patients with first episode psychosis (FEP), but it remains unclear whether those predictions generalize to independent populations. Using a subset of demographic and clinical baseline predictors, we aimed to develop and externally validate different models predicting functional outcome after a FEP in the context of a schizophrenia-spectrum disorder (FES), based on a previously published cross-validation and machine learning pipeline. A crossover validation approach was adopted in two large, international cohorts (EUFEST, n = 338, and the PSYSCAN FES cohort, n = 226). Scores on the Global Assessment of Functioning scale (GAF) at 12 month follow-up were dichotomized to differentiate between poor (GAF current < 65) and good outcome (GAF current ≥ 65). Pooled non-linear support vector machine (SVM) classifiers trained on the separate cohorts identified patients with a poor outcome with cross-validated balanced accuracies (BAC) of 65-66%, but BAC dropped substantially when the models were applied to patients from a different FES cohort (BAC = 50-56%). A leave-site-out analysis on the merged sample yielded better performance (BAC = 72%), highlighting the effect of combining data from different study designs to overcome calibration issues and improve model transportability. In conclusion, our results indicate that validation of prediction models in an independent sample is essential in assessing the true value of the model. Future external validation studies, as well as attempts to harmonize data collection across studies, are recommended.

Authors

Institutions

Publication Details

Journal
Schizophrenia
Published
2024-10-07
DOI
https://doi.org/10.1038/s41537-024-00505-w
Citations
5
Primary Topic
Schizophrenia research and treatment
Type
article
Field-Weighted Citation Impact
1.93

Funders

Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

Multivariable prediction of functional outcome after first-episode psychosis: a crossover validation approach in EUFEST and PSYSCAN

Celso Arango, George Gifford, Gemma Modinos, Anke Maatz et al.
5 citations
Schizophrenia
Schizophrenia research and treatment
1.93
article

Multivariable prediction of functional outcome after first-episode psychosis: a crossover validation approach in EUFEST and PSYSCAN

Celso Arango, George Gifford, Gemma Modinos, Anke Maatz, Graccielle R. Cunha, Dominic Oliver, Emily Hird, Matthew J. Kempton, Christos Pantelis, Rosa Ayesa‐Arriola, Jenny Lepock, Stefan Kaiser, Yoav Domany, Cory Gerritsen, Paola Dazzan, Romina Mizrahi, Giuseppe Piegari, Sandra Vieira, Roberto Rodríguez–Jiménez, Linda Levi, Mathilde Antoniades, Machteld Marcelis, Mark Weiser, Brian P. Hallahan, Wolfram Kawohl, Jun Soo Kwon, Kate Merritt, Mara Parellada, Julia Furtner, Dominic Dwyer, Matthäus Willeit, Matthias Kirschner, Florian Bitsch, Meredith McHugh, Oliver Gruber, René S. Kahn, Benedicto Crespo‐Facorro, Silvana Galderisi, Margot I. E. Slot, Covadonga M. Díaz‐Caneja, Armida Mucci, Minah Kim, Alexis E. Cullen, Natalia Petros, Jessica Spark, Barnaby Nelson, Marina Díaz‐Marsá, Joost Janssen, Claudia Vingerhoets, Anja Richter, Lieuwe de Haan, Ana Weidenauer, Alessia Nicita, Tae Young Lee, Miriam Ayora, Hendrika H. van Hell, Silvia Kyungjin Lho, Birte Yding Glenthoj, Víctor Ortiz‐García de la Foz, Daria Pietrafesa, Patrick D. McGorry, Paolo Fusar‐Poli, Dara M. Cannon, Fiona Coutts, Michael Kiang, Sara Patriarca, Egill Rostrup, Andrea Mechelli, Philipp Stämpfli, Ivana Prce, Jayachandra M. Raghava, Stephen M. Lawrie, Tilo T. J. Kircher, Mikkel Sørensen, Thomas Spencer, Gabriele Sachs, Bea Campforts, James McLoughlin, Andrea De Micheli, Helen Baldwin, G. Paul Amminger, Bjørn Hylsebeck Ebdrup, Colm McDonald, Esther Setién‐Suero, Martha Finnegan, Erika van Hell, André Zugman, Irina Falkenberg, Bernd Krämer, Ary Gadelha, Daniela Prayer, Kang Ik Kevin Cho, U Sauerzopf, Philip K. McGuire, Jens Sommer, Margaret Maheandiran, Achim Burrer, Inge Winter, Paula Suárez‐Pinilla, Jessica Merchán‐Naranjo, Diana Tordesillas‐Gutiérrez, Rodrigo A. Bressan, Stefania Tognin, Frederike Schirmbeck, Nikolaos Koutsouleris, Sun-Young Moon, Paola Bucci, Marcena Lenczowska, A. Ter Maat, W.W. Fleischhacker, Inge Winter van Rossum, Zhaoying Yu, Sarah Ahmed, Naemi Huber, Maria F. Urquijo Castro
article en
5 citations

Abstract

Several multivariate prognostic models have been published to predict outcomes in patients with first episode psychosis (FEP), but it remains unclear whether those predictions generalize to independent populations. Using a subset of demographic and clinical baseline predictors, we aimed to develop and externally validate different models predicting functional outcome after a FEP in the context of a schizophrenia-spectrum disorder (FES), based on a previously published cross-validation and machine learning pipeline. A crossover validation approach was adopted in two large, international cohorts (EUFEST, n = 338, and the PSYSCAN FES cohort, n = 226). Scores on the Global Assessment of Functioning scale (GAF) at 12 month follow-up were dichotomized to differentiate between poor (GAF current < 65) and good outcome (GAF current ≥ 65). Pooled non-linear support vector machine (SVM) classifiers trained on the separate cohorts identified patients with a poor outcome with cross-validated balanced accuracies (BAC) of 65-66%, but BAC dropped substantially when the models were applied to patients from a different FES cohort (BAC = 50-56%). A leave-site-out analysis on the merged sample yielded better performance (BAC = 72%), highlighting the effect of combining data from different study designs to overcome calibration issues and improve model transportability. In conclusion, our results indicate that validation of prediction models in an independent sample is essential in assessing the true value of the model. Future external validation studies, as well as attempts to harmonize data collection across studies, are recommended.

SchizophreniaVol. 10(1)
University of Copenhagen (DK), Brigham and Women's Hospital (US), Universidad de Cantabria (ES), Centre for Addiction and Mental Health (CA), Harvard University (US), Seoul National University (KR), Innsbruck Medical University (AT), Philipps University of Marburg (DE), Tel Aviv University (IL), The University of Melbourne (AU), Queen Mary University of London (GB), King's College London (GB), University of Toronto (CA), Ollscoil na Gaillimhe – University of Galway (IE), Universität Innsbruck (AT), Utrecht University (NL), University of Campania "Luigi Vanvitelli" (IT), South London and Maudsley NHS Foundation Trust (GB), University of Zurich (CH), Heidelberg University (DE), University of Pavia (IT), Sheba Medical Center (IL), Instituto de Salud Carlos III (ES), Seoul Metropolitan Government (KR), Royal Edinburgh Hospital (GB), Canada Research Chairs (CA), Glostrup Hospital (DK), Rigshospitalet (DK), Seoul National University Hospital (KR), Seoul National University Bundang Hospital (KR), Warneford Hospital (GB), Karolinska Institutet (SE), Hospital General Universitario Gregorio Marañón (ES), Hospital Clínico San Carlos (ES), University Medical Center Utrecht (NL), Maastricht University (NL), University of Oxford (GB), Centro de Investigación Biomédica en Red de Salud Mental (ES), Chonnam National University Hospital (KR), Instituto de Investigación Sanitaria del Hospital Clínico San Carlos (ES), Marqués de Valdecilla University Hospital (ES), SNUH SMG-SNU Boramae Medical Center (KR), University Hospital of Geneva (CH), Orygen (AU), Instituto de Investigación Marqués de Valdecilla (ES), Max Planck Institute of Psychiatry (DE), Amsterdam University Medical Centers (NL), McGill University (CA), Medical University of Vienna (AT), Ludwig-Maximilians-Universität München (DE), Universidade Federal de São Paulo (BR), University of Amsterdam (NL), University of Edinburgh (GB), Icahn School of Medicine at Mount Sinai (US)
Pfizer, AstraZeneca, Sanofi
Openalex Percentile: Top 13%
Schizophrenia research and treatment
1.93
DOIOpen Access PDFOpenAlex
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.