Lightweight multi-branch feature fusion network for multimodal medical image fusion

Most existing medical image fusion methods, which are based on deep learning achieve satisfactory results by using complex network architectures and stacking numerous modules. However, these methods often overlook the application scenarios of multimodal medical image fusion. The complex model structure and large parameter load make deployment on mobile devices extremely challenging. Moreover, it is unreasonable to consume substantial computational resources at the low-level image processing stage if the method is to be applied to downstream computational tasks. We have innovatively designed a lightweight multi-branch feature fusion network for multimodal medical image fusion. This method has a lower parameter count and extremely fast forward inference speed. This is due to our designed multi-branch feature channel segmentation method, which divide-and-conquer feature extraction with different receptive fields. We also use a channel attention mechanism based on contrast awareness to fuse and reduce the dimensionality of the feature maps, preserving ample source image information while reducing computational load. Finally, the fusion image reconstruction is completed through a sliding window attention mechanism combined with long-range feature dependencies.