Integrating self-attention transformers and multi-dimensional approaches for multi-modal medical image fusion

Due to the progress of artificial intelligence (AI), researchers in the field of data and computer vision are highly intrigued by the potential application of AI in medical image fusion. Multiple imaging modalities can be combined to improve patient care, therapeutic planning, and medical diagnosis via image fusion technologies. This study intends to assess the effectiveness of the transformer in medical image fusion through an attention-based fusion method. The proposed model, M3IFT1.0, is an advanced AI system that employs transformers to automatically retrieve features from clinical images. It also introduces a unique attention-based fusion rule that enhances the effectiveness and robustness of medical image fusion by applying the L2 norm. The proposed model was trained on the publicly available MS COCO dataset and employed to make predictions on the Lung dataset and validated on the Whole Brain Atlas dataset. The quality of the fused output images is assessed on 13 performance metrics. The proposed system is evaluated and compared to both traditional techniques and recent systems based on deep learning. The results demonstrate that the proposed model performs at least 24.7% better than all conventional and deep learning techniques used for benchmarking.