CausalExplain: Causal Explanations of Black-box Models with Training Data Subsets

AI models are becoming increasingly opaque, prompting significant efforts to explain their predictions. However, data-based explanations-i.e., explaining predictions using training data points-remain largely underexplored. Such explanations are essential for debugging training data and gaining deeper insights into model behavior. We introduce a demo of CausalExplain, a model-agnostic system designed to trace the predictions of popular machine learning (ML) models back to their corresponding training data points. For any given prediction, CausalExplain provides explanations based on predicates from the training data that caused the outcome. These explanations are derived from the causal DAG of the training data, with CausalExplain presenting the user with the top-k data-centric explanations for a given query row. In this demo, participants will explore how causal DAGs support the generation of such explanations across various datasets and underlying ML models.