Spatial and learnable frequency dynamic collaborative visual perception network for remote sensing images semantic segmentation

Remote sensing semantic segmentation , as a current research hotspot, is widely applied in scenarios such as agricultural planning and urban construction planning . Deep learning models based on spatial and frequency domain co-perception have rapidly developed due to their comprehensive perception capabilities. However, current networks primarily apply frequency domain perception within various attention mechanisms , lacking complementary learning with the spatial domain. In addition, whether the encoder–decoder pairing in existing U-shaped architectures can fully exploit the advantages of dual-domain perception requires further discussion. To address these issues, this paper proposes the Spatial and Learnable Frequency Dynamic Collaborative Visual Perception Network (SFCVPNet). The encoder uses a cascade of Spatial and Frequency Domain-Aware Transformer (SFFormer) blocks and Convolutional Neural Networks (CNNs) blocks to perceive features, while the decoder employs simple skip connections, forming a novel integration method of CNNs and Transformer blocks in the U-shaped architecture. SFFormer includes the Spatial and Frequency Domain Collaborative Perception Module (SFCPM) and the Spatial and Frequency Guided Multi-Layer Perceptron (SF-MLP). Both SFCPM and SF-MLP utilize a dual-domain perception approach to extract and integrate features. Additionally, we adopt learnable operations for frequency domain feature perception, ensuring that frequency domain features meet the learning requirements of the network model. On three public datasets: ISPRS Vaihingen, ISPRS Potsdam, and LoveDA, respectively, we achieved mIoU scores of 85.90%, 88.19%, and 55.6%. The code will be released at https://github.com/cslxju/SFCVPNet . • SFCVPNet is an end-to-end UNet semantic segmentation framework. • The proposed SFCVPNet method achieves global-local information fusion. • SF-MLP introduces frequency information to assist feature learning. • The experiment demonstrated superior semantic segmentation performance.