LBFormer: Scene Perception Segmentation Transformer Based on Local Block

Scene perception for autonomous vehicles and vessels is crucial for autonomous navigation. Current mainstream transformer methods typically split the feature map into windows, such as local, dilated, and horizontal/vertical bar windows. However, their token interaction is confined to fixed windows, posing challenges for image-based semantic segmentation. This article proposes a novel model, LBFormer, which enables flexible token interaction across different windows. Specifically, a window-level affinity graph is constructed from coarse-grained features using self-attention clustering and evolves during training, retaining top-k windows with high semantic relevance for each window. Self-attention purification is then employed to compress and filter fine-grained features with low semantic relevance within the top-k windows, ensuring effective token interaction for each feature point. To enhance context modeling within windows and build a more effective window-level affinity graph, a dual branch method extracts multidimensional features from each window, which are then interacted with and fused via the feature aggregation module. Extensive experiments at an image resolution of 224×224 were conducted on our private YZ-DATA water surface scene dataset and the public CamVid urban scene dataset. The results show that LBFormer achieves an MIoU of 89.80% on YZ-DATA and 61.31% on CamVid, surpassing mainstream transformer methods.