Analysing the Learning of Interpretable Features from Histopathological Images for Breast Cancer Classification Using Saliency Maps

This paper investigates the potential of saliency maps to interpret deep learning models trained on Hematoxylin and Eosin (H&E) stained images for breast cancer classification. The study utilizes three convolutional neural networks-VGG16, ResNet50, and InceptionV3-comparing the saliency maps generated by GradCAM++ with their corresponding Immunohistochemical (IHC) images, which serve as ground truth interpretable features. Various metrics, including Intersection over Union (IoU), Dice Coefficient, Balanced Accuracy, and Matthews Correlation Coefficient (MCC), were employed to assess the alignment between the saliency maps and the IHC images. The models were trained from scratch and using transfer learning with ImageNet weights to evaluate the effectiveness of transfer learning in this context, respectively. Experimental results indicate that transfer learning enhances a model's performance, particularly for ResNet50. The study concludes that IoU and Dice Coefficient are the most reliable metrics for comparing saliency maps with IHC images, while Balanced Accuracy and MCC may fail to detect certain anomalies.