Explainable AI Methods in Medical Image Analysis

This paper presents a systematic review of explainable artificial intelligence (XAI) methods applied to COVID-19 image analysis. We examined 154 records published between 2020 and 2024, identifying 45 distinct XAI techniques. Our analysis shows that a limited number of methods, primarily Grad-CAM, LIME, and SHAP, were frequently employed. While these techniques improve transparency in deep learning models, most studies rely on a single approach rather than integrating multiple methods to offer comprehensive interpretability. The findings highlight significant progress in making Deep Learning models more understandable for clinical decision-making, while also emphasizing the need for further research to combine and refine these techniques. This work aims to guide future efforts in developing transparent models for medical imaging applications.