Towards Explainable Image Fusion: Gradient-Based Heatmaps for Modal Contributions

While explainability methods are common in image classification, there's a notable lack of approaches tailored for understanding image fusion. This paper tackles the need for interpretability in deep learning-based image fusion methods. We introduce a novel framework that uses gradient-based heatmaps to visualize the contributions of each modality in multi-sensor fusion. Using the ConvFuse model as a case study, our method generates distinct heatmaps for visible and thermal images, making it easy to see which regions shape the model's output. A combined heatmap further reveals pixel-level modality prioritization, showing how each image part contributes to the fusion process. To assess the impact of the interconnected architecture of the ConvFuse model, we also apply our framework to a non-interconnected variant, comparing results to validate the benefits of its structural design. Adaptable to other fusion models, this framework enhances transparency and builds trust in AI by making complex decisions easier to interpret, especially in critical applications.