Unsupervised cross-modal biomedical image fusion framework with dual-path detail enhancement and global context awareness

Fluorescence imaging and phase-contrast imaging are two important imaging techniques in molecular biology research. Green fluorescent protein images can locate high-intensity protein regions in Arabidopsis cells, while phase-contrast images provide information on cellular structures. The fusion of these two types of images facilitates protein localization and interaction studies. However, traditional multimodal optical imaging systems have complex optical components and cumbersome operations. Although deep learning has provided new solutions for multimodal image fusion, existing methods are usually based on convolution operations, which have limitations such as ignoring long-range contextual information and losing detailed information. To address these limitations, we propose an unsupervised cross-modal biomedical image fusion framework, called UCBFusion. First, we design a dual-branch feature extraction module to retain the local detail information of each modality and prevent the loss of texture details during convolution operations. Second, we introduce a context-aware attention fusion module to enhance the ability to extract global features and establish long-range relationships. Lastly, our framework adopts an interactive parallel architecture to achieve the interactive fusion of local and global information. Experimental results on Arabidopsis thaliana datasets and other image fusion tasks indicate that UCBFusion achieves superior fusion results compared with state-of-the-art algorithms, in terms of performance and generalization ability across different types of datasets. This study provides a crucial driving force for the development of Arabidopsis thaliana research.