Failure Modes of Prediction Models Built from Electronic Health Record Data: Methods and Preliminary Results of the First Pass of a Scoping Review
This report records the methods and preliminary results of the first pass of a scoping review of failure modes of prediction models built from electronic health record (EHR) data. A failure mode is a distinct way in which a failure can arise at any stage of a prediction project. A codebook of 80 anticipated failure modes in 14 families was built from three independent sources: a process map derived from a draft theory of how EHR data are created and reused, reporting standards and appraisal tools, and a random sample of the literature. PubMed was searched for records added on or before 23 September 2026, and 340 records were screened on title and abstract by one reviewer with advisory labels from a large language model. In all, 192 papers were provisionally included (86 central, 32 peripheral and 74 on mitigation). They mention 53 of the 80 anticipated failure modes, 47 of them in a central paper, and suggest seven provisional new modes. An earlier mapping of the failure modes onto the draft process map suggested that it could explain most of them at least in part and identified where it falls short. All results are preliminary and will be revised when the full texts are read in dedicated reviews of each family.
Authors
- Uday Pratap Singh (ORCID: https://orcid.org/0009-0003-0026-767X)
Publication Details
- Journal
- Open Science Framework
- Published
- 2026-11-12
- DOI
- https://doi.org/10.17605/osf.io/3vr8j
- Primary Topic
- Machine Learning in Healthcare
- Type
- article
- Field-Weighted Citation Impact
- 0.00