PROBAST+AI: an updated quality, risk of bias, and applicability assessment tool for prediction models using regression or artificial intelligence methods
The Prediction model Risk Of Bias ASsessment Tool (PROBAST) is used to assess the quality, risk of bias, and applicability of prediction models or algorithms and of prediction model/algorithm studies. Since PROBAST’s introduction in 2019, much progress has been made in the methodology for prediction modelling and in the use of artificial intelligence, including machine learning, techniques. An update to PROBAST-2019 is thus needed. This article describes the development of PROBAST+AI. PROBAST+AI consists of two distinctive parts: model development and model evaluation. For model development, PROBAST+AI users assess quality and applicability using 16 targeted signalling questions. For model evaluation, PROBAST+AI users assess the risk of bias and applicability using 18 targeted signalling questions. Both parts contain four domains: participants and data sources, predictors, outcome, and analysis. Applicability of the prediction model is rated for the participants and data sources, predictors, and outcome domains. PROBAST+AI may replace the original PROBAST tool and allows all key stakeholders (eg, model developers, AI companies, researchers, editors, reviewers, healthcare professionals, guideline developers, and policy organisations) to examine the quality, risk of bias, and applicability of any type of prediction model in the healthcare sector, irrespective of whether regression modelling or AI techniques are used.
Authors
- Constanza L. Andaur Navarro (ORCID: https://orcid.org/0000-0002-7745-2887)
- Karandeep Singh (ORCID: https://orcid.org/0000-0001-8980-2330)
- Johannes B. Reitsma (ORCID: https://orcid.org/0000-0003-4026-4345)
- André Pascal Kengne (ORCID: https://orcid.org/0000-0002-5183-131X)
- Patrícia Logullo (ORCID: https://orcid.org/0000-0001-8708-7003)
- Lena Maier‐Hein (ORCID: https://orcid.org/0000-0003-4910-9368)
- Leo Anthony Celi (ORCID: https://orcid.org/0000-0001-6712-6626)
- Daniel Shu Wei Ting (ORCID: https://orcid.org/0000-0003-2264-7174)
- Georg Heinze (ORCID: https://orcid.org/0000-0003-1147-8491)
- Richard D Riley (ORCID: https://orcid.org/0000-0001-8699-0735)
- Xiaoxuan Liu (ORCID: https://orcid.org/0000-0002-1286-0038)
- Lotty Hooft (ORCID: https://orcid.org/0000-0002-7950-2980)
- Bada Yang (ORCID: https://orcid.org/0000-0002-9317-4995)
- Paula Dhiman (ORCID: https://orcid.org/0000-0002-0989-0623)
- Professor Gary S. Collins (ORCID: https://orcid.org/0000-0002-2772-2316)
- Melissa D. McCradden (ORCID: https://orcid.org/0000-0002-6476-2165)
- Johanna AAG Damen (ORCID: https://orcid.org/0000-0001-7401-4593)
- Maarten van Smeden (ORCID: https://orcid.org/0000-0002-5529-1541)
- Lauren Oakden‐Rayner (ORCID: https://orcid.org/0000-0001-5471-5202)
- Laure Wynants (ORCID: https://orcid.org/0000-0002-3037-122X)
- Marzyeh Ghassemi (ORCID: https://orcid.org/0000-0001-6349-7251)
- Spiros Denaxas (ORCID: https://orcid.org/0000-0001-9612-7791)
- Alastair Keith Denniston (ORCID: https://orcid.org/0000-0001-7849-0087)
- Andrew L. Beam (ORCID: https://orcid.org/0000-0002-6657-2787)
- Karel G.M. Moons (ORCID: https://orcid.org/0000-0003-2118-004X)
- Ben Van Calster (ORCID: https://orcid.org/0000-0003-1613-7450)
- T. K. Kaul (ORCID: https://orcid.org/0000-0002-4402-5379)
- Nan Liu (ORCID: https://orcid.org/0000-0003-3610-4883)
Institutions
- University Hospitals Birmingham NHS Foundation Trust (GB)
- Harvard University (US)
- University of Cape Town (ZA)
- German Cancer Research Center (DE)
- Utrecht University (NL)
- Heidelberg University (DE)
- University of Michigan (US)
- Hospital for Sick Children (CA)
- Nuffield Orthopaedic Centre (GB)
- University Medical Center Utrecht (NL)
- University College Birmingham (GB)
- British Heart Foundation (GB)
- Maastricht University (NL)
- University of Oxford (GB)
- National Center for Tumor Diseases (DE)
- Duke-NUS Medical School (SG)
- Australian Centre for Robotic Vision (AU)
- NIHR Birmingham Biomedical Research Centre (GB)
- University College London (GB)
- The University of Adelaide (AU)
- Massachusetts Institute of Technology (US)
- Medical University of Vienna (AT)
- University of Birmingham (GB)
- KU Leuven (BE)
Publication Details
- Journal
- BMJ
- Published
- 2025-03-24
- DOI
- https://doi.org/10.1136/bmj-2024-082505
- Citations
- 208
- Primary Topic
- Machine Learning in Healthcare
- Type
- article
- Field-Weighted Citation Impact
- 516.04
Funders
- Engineering and Physical Sciences Research Council