Quantifying Explainability: Essential Guidelines for Evaluating XAI Methods

Deep Learning (DL) models have demonstrated excellence in executing complex tasks within sensitive fields such as healthcare, finance, and autonomous systems. However, ensuring that their decisions are understandable and trustworthy has become a critical concern. Explainable AI methods (XAI) have emerged to address this challenge by offering diverse techniques to explain how DL models arrive at their predictions. The increasing number of XAI methods, combined with their varied outputs, makes it challenging to assess which method is most suitable for a given task. In the literature, two main approaches have been proposed for evaluating XAI methods: qualitative and quantitative. In this paper, we focus on quantitative metrics and essential properties, such as faithfulness, robustness, and complexity, to effectively evaluate XAI methods. By analyzing these properties, we provide clear insight into the advantageous values, whether higher or lower. Additionally, to ensure consistency across metrics, we propose a new scores' transformation so that higher values consistently indicate better performance. This work serves as a comprehensive guide for practitioners and researchers, enabling them to select, assess, and improve explainability methods tailored to specific use cases.