BaAFN: A Boundary-Aware Attention Fusion Network for Remote Sensing Semantic Segmentation

The performance of remote sensing semantic segmentation on object boundaries and small objects continues to pose a significant challenge due to semantics near them being complex and ambiguous. In this work, we propose the Boundary-aware Attention Fusion Network (BaAFN) including the Feature Attention Fusion Module (FAFM) from a representation optimization perspective and a Semantic-aware Contrastive Boundary Loss (SaCBL) from a boundary supervision perspective. Specifically, FAFM learns transformation offsets at the pixel level, facilitating contextual alignment of lower-level and higher-level features, and then rescales feature maps more comprehensively by capturing complementary global and local information across adjacent features to diminish the discrepancy in spatial and channel dimensions. SaCBL utilizes semantic relationships to mitigate the impact of inter-class high similarity or intra-class low similarity on edge prediction and aligns with the ground truth by the flow-based method that avoids conflicts with the optimization goal of the segmentation task. The FAFM and SaCBL are plug-and-play methods that can be integrated into any segmentation model to improve performance, especially for small objects and object boundaries. Extensive experimental evaluations demonstrate that our proposed method outperforms corresponding baselines and achieves state-of-the-art results on segmentation tasks. Especially, the proposed method can capture more detailed information, particularly edges, within complex scenarios. All the codes are available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/jiaenchen2024/BaAFN</uri>.