ResSCFuse: A Fusion Network for Infrared and Visible Images Based on Restormer Attention and Semi-Coupled Feature Reconstruction Net

The aim of fusing infrared and visible images is to produce a composite image that features both significant targets and detailed textures. The existing CNN-based and AutoEncoder(AE) fusion methods are not effective at capturing information from distant pixels, and when performing feature modeling within the same scene, they overlook the shared and private characteristics of cross-modal features. In this paper, we present a new approach for fusing infrared and visible images. The method utilizes a multi-scale features fusion structure, which comprise an encoder network, Semi-Coupled Feature Reconstruction Nets, fusion layers, and a decoder network. The encoder based on Restormer attention mechanism can extract multi-scale global-local features from the source images. Features at different scales are processed by the Semi-Coupled Feature Reconstruction Network (SCFRNet) to extract shared and private features. Then these features are input into a fusion layer for convolutional fusion. This allows for effective reconstruction of images from the feature maps. Experiments on public datasets show that this method outperforms other advanced fusion techniques methods in both objective and subjective assessments.