A Novel Graph-Based Framework for Understanding of Decision-Making Process in Deep Learning Models

In the realm of deep learning, the lack of transparency in complex models is a major obstacle for wider adoption and trust in artificial intelligence systems. Current explainability methods often provide either localized or global insights into model decisions, but rarely offer a unified perspective that combines both. To address this gap, we propose a novel framework that integrates multiple interpretability techniques, GradCAM and Integrated Gradients to offer a comprehensive, multi-perspective explanation of deep learning models in terms of graphs. Our framework initially identifies key regions of interest in the input data offering better understanding of model behaviour. The salient regions are then mapped into a graph-based representation, where nodes represent clusters of important features, and edges represent the intensity of their interactions. This dual-layer approach provides both intuitive visual heatmaps for localized insights and graph structures for understanding feature interactions. The aim of this work is to improve model transparency and interpretability by combining visual and structural explanations, thus enabling a more comprehensive understanding of complex deep learning models. To validate the proposed framework, we conduct experiments using pre-trained networks on image classification dataset. Our results demonstrate that the proposed explainability framework not only enhances interpretability by highlighting critical regions but also offers novel insights into the relationships between features through graph structures. Our code is available at https://github.com/pubudusanjeewani/arcdp-2021-2024/tree/main/ssci2025