NCSBFF-Net: Nested Cross-Scale and Bidirectional Feature Fusion Network for Lightweight and Accurate Remote-Sensing Image Semantic Segmentation

Semantic segmentation has emerged as a critical research area in Earth observation. This paper proposes a novel end-to-end semantic segmentation network, the Nested Cross-Scale and Bidirectional Feature Fusion Network (NCSBFF-Net), to address issues such as intra-class heterogeneity, inter-class homogeneity, scale variability, and the classification of tiny objects. Specifically, a CNN-based lightweight feature pyramid module is employed to extract contextual information across multiple scales, thereby addressing intra-class heterogeneity. The NCSBFF module leverages features from both shallow and deep layers and is designed to fuse multi-scale features, thereby enhancing inter-class semantic differences. Additionally, the shallowest feature is passed to the Shuffle Attention block in the NCSBFF module, which adaptively filters out weak details and highlights critical information for the classification of tiny objects. Extensive experiments were conducted on the Potsdam and Vaihingen benchmarks. Experiment results demonstrate that the NCSBFF-Net outperforms state-of-the-art methods, achieving a better trade-off between accuracy and efficiency, with a 5% improvement in mIoU significantly enhancing the recognition capability of small and complex objects, such as vehicles and irregular land parcels, in challenging scenes, and a 1.73% increase in accuracy demonstrating a better balance between computational efficiency and segmentation accuracy, providing an optimized solution for deployment on edge devices.