Modelling radiological features fusion and explainable AI in pneumonia detection: A graph- based deep learning and transformer approach

• Pneumonia causes 2.5 million deaths yearly, with children under five significantly affected globally. • Deep learning-based methodology addresses challenges in diagnosing pneumonia using annotated lung CT scans. • U-Net achieved 0.943 Dice Similarity Coefficient and 0.89 IoU for effective lung segmentation accuracy. • Proposed model demonstrated 94% accuracy with strong precision, recall, and F1-scores in classification tasks. • Integration of graph-based representations, transformers, and radiological fusion enhances pneumonia detection model efficiency. This study aims to develop a deep learning-based methodology for pneumonia detection using lung CT scans. The research focuses on integrating U-Net for lung segmentation, graph-based feature representations, and transformer-based models to improve interpretability and diagnostic accuracy. We utilized 500 annotated lung CT scans, split into 70% training (350 images), 15% validation (75 images), and 15% testing (75 images). U-Net was employed for lung segmentation, with segmentation performance evaluated using Dice Similarity Coefficient (DSC) and Intersection over Union (IoU). The classification model incorporated feature fusion, combining handcrafted radiological features with transformer-based feature extraction. The model’s noise sensitivity and interpretability were assessed using Grad-CAM and attention maps. The segmentation model achieved a DSC of 0.943 on the training set, 0.928 on the validation set, and 0.910 on the test set. The IoU values were 0.890, 0.872, and 0.850, respectively. The model’s inference time was 25 milliseconds per image. The classification model achieved an overall accuracy of 94%, with pneumonia cases showing precision, recall, and F1-scores of 0.95, 0.94, and 0.94, respectively. For normal cases, precision, recall, and F1-scores were 0.92, 0.94, and 0.93. The model’s performance improved over 20 epochs, with training accuracy reaching 0.94 and validation accuracy reaching 0.88. The feature fusion method achieved an attention score of 0.88, with node intensity scores of 0.85. Noise sensitivity analysis revealed a performance drop beyond a noise level of 0.3. This study demonstrates the potential of AI-driven solutions for pneumonia detection, achieving high accuracy, robustness, and interpretability. While the model performs well, further optimization is needed to address noise sensitivity and enhance model generalization for real-world applications.