Dual-Branch Network with Transformer and CNN for Underwater Semantic Segmentation

Underwater semantic segmentation provides visual information for underwater exploration and surveying applications, playing a crucial role in the development and conservation of marine resources. However, semantic segmentation of underwater images is technically challenging due to the complexity and challenging nature of underwater environments. In this work, we propose an underwater semantic segmentation network based on convolutional neural networks (CNNs) and Transformers to extract comprehensive features of underwater objects from both global and local perspectives. To address the scale variations of underwater objects, we propose a multi-scale feature enhancement module (MSFE), which utilizes multiple receptive fields to adapt to underwater objects at different scales. In addition, a local feature extraction (LFE) module is introduced to capture underwater objects of different shapes, which adaptively fuses conventional and deformable convolutional features using the spatial attention mechanism. Experimentally, our model outperforms previous semantic segmentation methods on the underwater semantic segmentation benchmark datasets, i.e., the SUIM dataset, and the DeepFish dataset. In detail, the mean intersection over union (mloU) and mean pixel accuracy (mPA) of our model on the SUIM dataset achieve $\mathbf{7 4. 2 1} \%$ and $\mathbf{8 1. 6 7 \%}$, $\mathbf{2. 4 4 \%}$ and $\mathbf{2. 9 1 \%}$ higher than the baseline model, respectively. And for the DeepFish dataset, our model achieves $\mathbf{9 6. 5 7 \%}$ and $\mathbf{9 7. 9 9} \%$ for mloU and $\mathbf{m P A}$, respectively.