AIM-X: Attention-Infused Multimodal Cross-Interaction for X-ray Clinical Report Generation

Medical imaging reports are a crucial part of the medical diagnosis process. The generation of reports from multimodal medical data includes images and text reports, which have been a potential area of clinical assistance in diagnostic tasks. In this work, we introduce a novel architecture, we called attention-infused multimodal cross-interaction (AIM-X), designed to extract and summarize information from X-ray images and their corresponding textual reports into coherent and comprehensive diagnostic reports. AIM-X model introduces a dual-attention mechanism to emphasize the most significant areas within the images, enabling precise diagnoses by healthcare professionals. AIM-X further reinforces the coherence and relevance of the generated reports, embedding the model with an explainability layer that enables visualization of the attention weights. The generated reports have a high proportion of extracting significant medical observations out of the ground truth accurately. Quantitatively, the performance of AIM-X was validated through BLEU measures of 0.7272 (BLEU1), 0.6595 (BLEU2), 0.6012 (BLEU3), 0.4686 (BLEU4), and a ROUGE value of 0.7272, representing a high level of similarity between generated reports and target diagnoses, achieving an accuracy of 94.5%.