Pixel Lens: A Granular Assessment of Saliency Explanations

ABSTRACT Deep neural networks have demonstrated exceptional performance across various domains. However, their black‐box nature poses significant challenges to understanding their decision‐making processes. As these models increasingly underpin critical decisions in regulated sectors, explainable artificial intelligence (XAI) has emerged as a promising avenue for elucidating model behaviour. Despite advancements, this field lacks standardised protocols for rigorously evaluating the correctness of explanation methods, particularly at a granular pixel level . This article introduces Pixel Lens, a novel protocol designed to assess the correctness of saliency‐based XAI methods at the pixel level. In contrast to approaches that rely on human‐centric semantically meaningful object assessments, Pixel Lens focuses on the actual decision rules employed by classifiers. The protocol systematically embeds spatially localised shortcuts into images, which are artificial patterns that networks preferentially adopt as decision rules, while preserving the original image features. After verifying that the shortcut is the dominant rule, Shapley values are estimated for pixels to produce ground‐truth explanation maps. Through extensive evaluations of saliency‐based XAI methods across multiple architectures and datasets, our experiments reveal that existing methods often fail to provide accurate pixel‐level attributions. These findings challenge recent progress claims in XAI and establish a more robust foundation for evaluating explanation techniques.