An Optical and SAR Sensor Image Fusion Method Based on Improved Auto-Encoder

There are significant differences between optical and synthetic aperture radar (SAR) sensor image data, and extracting complementary features useful for subsequent tasks from image data acquired by these two different types of sensors is a key issue. In this paper, we propose an optical and SAR sensor image fusion method based on improved Auto-Encoder (AE). The method employs a two-branch feature extraction network constructed using Convolutional Neural Network (CNN) and Transformer network to extract detail and base features of the image, respectively. Furthermore, Lite Transformer (LT) and DropKey are introduced to enhance the computational efficiency of the network and base feature extraction. The Convolutional Block Attention Module (CBAM) enables the CNN to extract detail features with greater efficiency. Finally, the concept of correlation is applied in the fusion stage, which facilitates the comprehensive extraction of base and detail features. The experimental results demonstrate that the algorithm proposed in this paper is capable of generating superior optical and SAR fused imagery.