Value of diversity characteristics in predictive modeling: ACS screening as a case study

We sought to improve the subgroup performance variability of a model that identifies arriving ED patients at high risk for ACS, to receive an ECG within 10 minutes of arrival, to detect STEMI. We compared a <i>Base Model</i> using age, sex, and chief complaint alone to (1) an <i>Interactions Model</i> adding interactions among the 3 variables, and (2) a <i>Diversity-Sensitive Model</i> including race, ethnicity, language, as well as identity interactions. We quantified human performance and combined it with each of the 3 models simulating use as practice augmenting AI predictions. With sensitivity as our primary outcome, we found humans at 72.8% were bested by the Diversity-Sensitive Model at 82.8%, and by the human-augmented Diversity-Sensitive Model at 91.3%, improving ACS predictions in all demographic subgroups. However, there was residual variation among subgroups (range of sensitivity: 62%-98%). Given risk distribution differences, subgroup-specific ECG-testing thresholds may further equitize ACS prediction performance.