CEERS: Counterfactual Evaluations of Explanations in Recommender Systems

The increasing focus on explainability within ethical AI, mandated by frameworks such as GDPR, highlights the critical need for robust explanation mechanisms in Recommender Systems (RS). A fundamental aspect of advancing such methods involves developing reproducible and quantifiable evaluation metrics. Traditional evaluation approaches involving human subjects are inherently non-reproducible, costly, subjective, and context-dependent. Furthermore, the complexity of AI models often transcends human comprehension capabilities, rendering it challenging for evaluators to ascertain the accuracy of explanations. Consequently, there is an urgent need for objective and scalable metrics that can accurately assess explanation methods in RS. Drawing inspiration from established practices in computer vision, this research introduces a counterfactual methodology to evaluate the accuracy of explanations in RS.