Semantic Segmentation of Multimodal Optical and SAR Images With Multiscale Attention Network

The joint semantic segmentation of multi-modal remote sensing images can make up for the problem of insufficient features of single-modal images and effectively improve the classification accuracy. Some deep learning methods have achieved good performance, but they face problems such as complex network structure, large number of parameters, and difficulty in deployment. In this paper, more attention is paid to front-end and branch-level feature transformation to obtain multi-scale semantic information. The multi-scale dilated extraction module (MDEM) is constructed to mine the specific features of different modalities. The multi-modal complementary attention module (MCAM) is designed for further acquiring prominent complementary content. The concatenated features are transmitted and reused by the dense convolution to complete the encoding. Ultimately, a general and concise end-to-end model is proposed. Comparative experiments are carried out on three heterogeneous datasets, and the model put forward performs well in qualitative analysis, quantitative comparison and visual effect. Meanwhile, the dexterity and practicability of the model are more prominent, which can provide support for lightweight design and hardware deployment.