Shapley value explanations for clinical prediction models: a scoping review and guide

Abstract Shapley value explanations are increasingly used to provide added transparency for “black-box” prediction models in healthcare. In this article, we provide an introductory overview of Shapley value explanations and describe their use through a detailed scoping review of 100 randomly selected, peer-reviewed publications that used Shapley explanations as part of their clinical research. We focus on the use of Shapley value explanations to explain the predictions of clinical prediction models that use tabular input features (e.g., predictors that measure patient characteristics like blood pressure and age) to predict health outcomes in individuals. In our review of the literature, we found that the methods used to compute Shapley value explanations were often underreported; e.g., 91% of publications did not disclose the algorithm that was used to compute Shapley value explanations, and 97% did not disclose the source or size of the reference (baseline) population used in the calculations. We identify four dominant motivations for using Shapley explanations in the literature (identification of key features, clinical decision support, trustworthiness, and exploratory analyses), elaborate on commonly found (mis) interpretations, and discuss challenges associated with Shapley value explanations. Finally, we provide practical recommendations for use of Shapley values in the context of clinical prediction models.