CALM-VLM: CALIBRATION AND SELECTIVE PREDICTION IN VISION–LANGUAGE MODELS FOR RELIABLE BRAIN MRI CLASSIFICATION
echanism for Vision-Language Models), which integrates confidence calibration and selective prediction into a generative 3D VLM. To create CALM-VLM, we fine-tuned the Med3DVLM architecture for Alzheimer's disease (AD) and stroke classification as initial test cases. To improve reliability, we incorporated temperature scaling on the VLM's generative outputs. The calibrated model then selectively abstained from predictions when uncertain; this also improved its diagnostic accuracy. Experiments across multi-site MRI datasets, from 10 countries worldwide, show that CALM-VLM improved confidence relative to uncalibrated VLMs. Coverage-adjusted test receiver-operator characteristic curve-area under the curve (ROC-AUC) increased by 5% to 13% for both diagnostic tasks across independent test sets. Our calibrated VLM achieved a test ROC-AUC of 0.951 for AD classification and 0.905 for stroke classification. These findings highlight the importance of calibrated, uncertainty-aware VLMs for trustworthy neuroimaging AI.
Authors
- María Mataró
- Fatemeh Geranmayeh (ORCID: https://orcid.org/0000-0003-1112-786X)
- Chirag Jagad (ORCID: https://orcid.org/0000-0002-6119-3956)
- George F. Wittenberg (ORCID: https://orcid.org/0000-0001-5603-1197)
- Steven A. Kautz (ORCID: https://orcid.org/0000-0003-3151-8235)
- Nicolas Schweighofer (ORCID: https://orcid.org/0000-0003-3362-6088)
- Nerses Sanossian (ORCID: https://orcid.org/0000-0002-8218-9518)
- Heidi M. Schambra (ORCID: https://orcid.org/0000-0002-1886-2288)
- Adrienne N. Dula (ORCID: https://orcid.org/0000-0002-4809-6115)
- Steven C. Cramer (ORCID: https://orcid.org/0000-0002-6214-6211)
- Fabrizio Piras (ORCID: https://orcid.org/0000-0003-3566-5494)
- Mahir H. Khan (ORCID: https://orcid.org/0000-0002-7215-9585)
- Abhishek Jaywant (ORCID: https://orcid.org/0000-0003-4307-5742)
- Na Jin Seo (ORCID: https://orcid.org/0000-0001-6446-5905)
- Surjo R. Soekadar (ORCID: https://orcid.org/0000-0003-1280-5538)
- Emily Rose Rosario (ORCID: https://orcid.org/0000-0002-1540-197X)
- Brenton Hordacre (ORCID: https://orcid.org/0000-0002-7129-6684)
- Kristin A. Wong (ORCID: https://orcid.org/0009-0004-0163-0466)
- Jessica M. Cassidy (ORCID: https://orcid.org/0000-0003-3469-0399)
- Nikhil J. Dhinagar (ORCID: https://orcid.org/0000-0003-2424-4854)
- Nerisa Banaj (ORCID: https://orcid.org/0000-0002-6317-8549)
- Pavithra Senthilkumar
- Chris M. Gregory
- Carolee Winstein
- Amy Brodtmann (ORCID: https://orcid.org/0000-0001-9466-2862)
- Adriana B. Conforto
- Sophia I. Thomopoulos
- Lara A. Boyd
- Gregory T. Thielman
- Kristan A. Leech
- Sook-Lei Liew
- Martin Lotze
- Michael R. Boric
- the ENIGMA-Stroke Recovery Working Group
- Paul M. Thompson
Institutions
- University of Southern California (US)
- Universitat Autònoma de Barcelona (ES)
- University of North Carolina Health Care (US)
- University of British Columbia (CA)
- Emory University (US)
- Medical University of South Carolina (US)
- University of Pittsburgh (US)
- College of Charleston (US)
- Cornell University (US)
- Universitätsmedizin Greifswald (DE)
- UCLA Health (US)
- Hospital Israelita Albert Einstein (BR)
- Casa Colina Centers for Rehabilitation (US)
- Ralph H. Johnson VA Medical Center (US)
- Fondazione Santa Lucia (IT)
- Imperial College London (GB)
- Saint Joseph's University (US)
- Monash University (AU)
- The University of Adelaide (AU)
- New York University (US)
- Universitat de Barcelona (ES)
- Charité - Universitätsmedizin Berlin (DE)
- The University of Texas at Austin (US)
Publication Details
- Journal
- bioRxiv (Cold Spring Harbor Laboratory)
- Published
- 2026-04-14
- DOI
- https://doi.org/10.64898/2026.04.10.717865
- Primary Topic
- Machine Learning in Healthcare
- Type
- article
- Field-Weighted Citation Impact
- 0.00