Explaining CNN classifications using small patches

Convolutional neural networks (CNNs) have achieved remarkable success in image classification tasks. However, their decision-making processes remain difficult to interpret, limiting their adoption in sensitive domains. We present here a novel explainability method that generates global explanations in the form of propositional rules, combining both pixel values and probabilities associated with sub-image patches. Our approach integrates a multi-layer perceptron trained on image patches, a CNN trained on patch probabilities, and a global rule extraction technique. The key idea is to highlight the most relevant image regions that the model uses for its predictions while maintaining high classification performance. We apply this method to three image classification problems: MNIST, CIFAR-10, and FER2013. The generated rulesets capture meaningful patterns in the data and provide accurate and faithful explanations. Although the rules generalize well on simpler datasets like MNIST, both simple and complex image classification problems, such as CIFAR-10, result in large rulesets, with the size increasing further as visual variability grows. Our method achieves competitive performance with standard CNNs while adding a rule-based explanation that highlights the inference process.