Toward trustworthy clinical AI for obsessive-compulsive disorder: reliability, generalizability, and interpretability of a transformer model across the ENIGMA-OCD consortium

BACKGROUND: Studies applying machine learning to obsessive-compulsive disorder (OCD) typically report accuracy in homogeneous samples but rarely assess model reliability, generalizability, and interpretability needed for clinical use. METHODS: We applied a transformer-based deep learning model, the Multi-Band Brain Net, to the ENIGMA-OCD cohort - the largest available resting-state functional magnetic resonance imaging (rs-fMRI) dataset in OCD with 1,706 participants (869 cases with OCD, 837 controls) across 23 sites worldwide. We evaluated model reliability by calculating calibration - the model's ability to "know what it doesn't know". We assessed generalizability using leave-one-site-out validation to test performance on unseen sites with different scanners, acquisition protocols, and patient populations. Finally, we examined interpretability by analyzing model attention weights to identify the neural connectivity patterns that influence model predictions. RESULTS: The model achieved modest but competitive classification performance (AUROC = .653, SD = .039). Crucially, while large-scale pretraining on the UK Biobank (N = 40,783) did not boost accuracy, it significantly enhanced model calibration by reducing overconfident predictions. Leave-one-site-out validation showed a generalization gap across sites (AUROC = .427-.819). Pretraining did not close this gap but removed scanner manufacturer bias. Finally, attention-based mapping identified biologically plausible patterns of widespread hypoconnectivity in OCD relative to healthy controls, particularly in low-frequency bands involving the default mode, salience, and somatomotor networks. These findings aligned with known OCD neurobiology. CONCLUSIONS: This study provides a framework for developing more reliable and trustworthy clinical artificial intelligence for OCD.

Authors

Institutions

Publication Details

Journal
medRxiv
Published
2026-04-27
DOI
https://doi.org/10.64898/2026.04.24.26351711
Primary Topic
Obsessive-Compulsive Spectrum Disorders
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

Toward trustworthy clinical AI for obsessive-compulsive disorder: reliability, generalizability, and interpretability of a transformer model across the ENIGMA-OCD consortium

Christopher Pittenger, J R Sato, Alan Anticevic, Federica Piras et al.
medRxiv
Obsessive-Compulsive Spectrum Disorders
article

Toward trustworthy clinical AI for obsessive-compulsive disorder: reliability, generalizability, and interpretability of a transformer model across the ENIGMA-OCD consortium

Christopher Pittenger, J R Sato, Alan Anticevic, Federica Piras, Pedro Morgado, Lea L. Backhausen, Yoshiyuki Hirano, Ysbrand D. van der Werf, Chris Vriend, Sophia I. Thomopoulos, S. Evelyn Stewart, Nadža Džinalija, Sara Bertolín, Susanne Walitza, Yoshinari Abe, Jun Soo Kwon, Emma Muñoz‐Moreno, Clara Moreau, Hirofumi Tomiyama, Fabrizio Piras, Silvia Brem, Marcelo Q. Hoexter, Jiook Cha, Marcelo C. Batistuzzo, Francesco Benedetti, Yuki Ikemizu, Jamie D. Feusner, Nora C. Vetter, Emily Stern, Patricia Gruner, Erika L Nurmi, Ganesan Venkatasubramanian, Chiang-Shan Ray Li, Ana Daniela Costa, Anders Lillevik Thorsen, Venkataram Shivakumar, Yuki Sakai, Carles Soriano‐Mas, Maria Picó‐Pérez, Srinivas Balachander, Kristen Hagen, Goi Khia Eng, Tokiko Yoshida, Philip R. Szeszko, Luisa Lázaro, Hein J. F. van Marle, José C. Pariente, Joseph O’Neill, Odile A. van den Heuvel, Dan Joseph Stein, Lea Waller, Fern Jaspers‐Fayer, Hanyang Ruan, Ignacio Martínez‐Zalacaín, Jonathan Ipser, José M. Menchón, Christine Löchner, Kathrin Koch, Je‐Yeon Yun, Daniela Rodriguez-Manrique, Pino Alonso, Zhen Wang, Paul M. Thompson, Rachel Marsh, Anouk van der Straten, Y Rangaswami Reddy, Maria Pak, Daniela Vecchio, Guido van Wingen, Youngsun Cho, Wieke van Leeuwen, Willem Bruin, H. Blair Simpson, Nicole Beyer, Eiji Shimizu, Sangyoon Bae, Youngchan Ryu, Minah Kim, Tomohiro Nakao, ENIGMA-OCD Working Group, Hao Hu, Janardhanan C. Narayanaswamy, Bjarne Hansen, Federica Colombo, Beatriz Couto, Qing Zhao, Benedetta Vai
article en

Abstract

BACKGROUND: Studies applying machine learning to obsessive-compulsive disorder (OCD) typically report accuracy in homogeneous samples but rarely assess model reliability, generalizability, and interpretability needed for clinical use. METHODS: We applied a transformer-based deep learning model, the Multi-Band Brain Net, to the ENIGMA-OCD cohort - the largest available resting-state functional magnetic resonance imaging (rs-fMRI) dataset in OCD with 1,706 participants (869 cases with OCD, 837 controls) across 23 sites worldwide. We evaluated model reliability by calculating calibration - the model's ability to "know what it doesn't know". We assessed generalizability using leave-one-site-out validation to test performance on unseen sites with different scanners, acquisition protocols, and patient populations. Finally, we examined interpretability by analyzing model attention weights to identify the neural connectivity patterns that influence model predictions. RESULTS: The model achieved modest but competitive classification performance (AUROC = .653, SD = .039). Crucially, while large-scale pretraining on the UK Biobank (N = 40,783) did not boost accuracy, it significantly enhanced model calibration by reducing overconfident predictions. Leave-one-site-out validation showed a generalization gap across sites (AUROC = .427-.819). Pretraining did not close this gap but removed scanner manufacturer bias. Finally, attention-based mapping identified biologically plausible patterns of widespread hypoconnectivity in OCD relative to healthy controls, particularly in low-frequency bands involving the default mode, salience, and somatomotor networks. These findings aligned with known OCD neurobiology. CONCLUSIONS: This study provides a framework for developing more reliable and trustworthy clinical artificial intelligence for OCD.

medRxiv
Universitat Jaume I (ES), University of Southern California (US), Nathan Kline Institute for Psychiatric Research (US), Johnson & Johnson (United States) (US), Centre for Addiction and Mental Health (CA), Kyushu University (JP), Seoul National University (KR), University of British Columbia (CA), Vita-Salute San Raffaele University (IT), University of Cape Town (ZA), Kyoto Prefectural University (JP), American Academy of Child and Adolescent Psychiatry (US), Stellenbosch University (ZA), Columbia University Irving Medical Center (US), Haukeland University Hospital (NO), Bellvitge University Hospital (ES), Centre Hospitalier Universitaire Sainte-Justine (CA), New Generation University College (ET), Shanghai Mental Health Center (CN), Seoul National University Hospital (KR), Yale University (US), New York Psychoanalytic Society and Institute (US), Humboldt-Universität zu Berlin (DE), Kyoto Prefectural University of Medicine (JP), Institut d'Investigació Biomédica de Bellvitge (ES), MSB Medical School Berlin (DE), University Hospital of Zurich (CH), Amsterdam Neuroscience (NL), Hospital Clínic de Barcelona (ES), James J. Peters VA Medical Center (US), Monash Health (AU), IRCCS Ospedale San Raffaele (IT), Hanyang University Seoul Hospital (KR), Hospital das Clínicas da Faculdade de Medicina da Universidade de São Paulo (BR), Arkin (NL), Fondazione Santa Lucia (IT), Reinier van Arkel (NL), Consorci Institut D'Investigacions Biomediques August Pi I Sunyer (ES), Johnson & Johnson (Brazil) (BR), Clinical Academic Center of Braga (PT), Center for Forensic Mental Health, Chiba University (JP), Klinik und Poliklinik für Psychotherapie und Psychosomatik (DE), Technical University of Munich (DE), National Institute of Mental Health and Neurosciences (IN), Universidade Federal do ABC (BR), Universitat de Barcelona (ES), Technische Universität Dresden (DE), Vrije Universiteit Amsterdam (NL), University of Minho (PT)
Openalex Percentile: Top 12%
Obsessive-Compulsive Spectrum Disorders
DOIOpen Access PDFOpenAlex
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.