From Image to Insight: Transformer-Based Multimodal Feature Fusion for Enhanced Chest X-Ray Report Generation

Automated chest X-Ray report generation offers a potential solution to reduce radiologists’ workload while improving efficiency and diagnostic accuracy. However, existing methods face challenges such as high visual similarity between normal and abnormal cases, limited descriptive diversity in reports, and biases in both textual and visual data. A novel framework has been proposed in this paper, combining multimodal fusion and contrastive learning to generate more descriptive and clinically relevant reports. The model integrates visual and textual features and employs mechanisms to focus on abnormal regions, enhancing report quality. Evaluated on the IU-X-Ray and MIMIC-CXR datasets, the framework achieves state-of-the-art performance in text generation (BLEU, ROUGE-L, METEOR) and clinical metrics (Precision, Recall, F1-Score). Human evaluation further validates its clinical utility, with fluency (3.62 and 3.61), faithfulness (3.90 and 3.92), and comprehensiveness (3.61 and 3.95) scores rated on a 5-point scale for the IU-X-Ray and MIMIC-CXR datasets, respectively. These results demonstrate the potential of the proposed framework for real-world clinical applications, enabling accurate and timely chest X-Ray report generation.