Correcting Biases of Shapley Value Attributions for Informative Machine Learning Model Explanations

Shapley value attribution (SVA) is an increasingly popular Explainable AI (XAI) approach that has been widely used in many recent applied studies to gain new insights into the underlying information systems. However, most existing SVA methods are error-prone, providing biased or unreliable explanations that fail to correctly capture the informational dependencies between features and model outputs. These explanation errors can be decomposed into two components: 1) observation bias which stems from data sparsity and leads to over-informativeness; and 2) structural bias which stems from distributional assumptions and leads to under-informativeness. To alleviate these biases, in this paper, we propose a series of refinement methods that combine out-of-distribution (OOD) detection and importance sampling. In essence, our methods aim to rectify the distribution drift caused by distributional assumptions. We apply our refinement methods to two popular SVAs: the marginal SVA and the surrogate model-based SVA. Our extensive experiments show that the proposed methods significantly enhance the informativeness of both local and global Shapley value-based explanations.