A Novel CAD Framework with Visual and Textual Interpretability: Multimodal Insights for Predicting Respiratory Diseases

Generating textual interpretability using recent advancements in large language models (LLMs) is crucial for enhancing the efficiency of comprehensive computer-aided diagnosis (CAD) systems. This improves transparency between medical staff, intelligent CAD systems, and end-users by creating a trustworthy and effective intermediate medical diagnosis environment. In this paper, an innovative explainable throughout CAD system is introduced, designed to predict diseases from Chest X-rays (CXR) in a comprehensive scenario. The primary goal is to undertake multiple tasks that reduce the burden on medical staff and enrich CAD outcomes, including classification, visual explanations (heatmaps), and textual report generation. The proposed CAD system is developed through eight key steps: Data Collection and Annotation, Data Preparation, Text Vectorizations (Indexing), Visual Encoder, RAG-Fusion, Structural Prompt, XAI LLmTextual Reasoning (LLM Model), and Final Output (LLM textual report, image classification, and heatmap localization). The AI-based CAD system is trained and evaluated using the public benchmark MIMIC-CXR dataset with 14 different classes. The classification performance achieved an overall accuracy of $70 \%$, precision of $70 \%$, and F1-score of $0.60 \%$, while for text report generation, the system obtained an average BERTScore precision of 0.83, RougeL 0.16, and a Meteor score of 0.28. These promising results suggest the potential for further improvement of the CAD system and its applicability to real-world medical tasks.