RDFormer: Efficient RGB-D Semantic Segmentation in Complex Outdoor Scenes

RGB-D data can now be easily acquired from vehicle-mounted sensors. This paper introduces RDFormer, an improved semantic segmentation method based on existing image segmentation approaches. The key innovation lies in combining self-attention mechanisms with heterogeneous data fusion. By applying self-attention to model global features across different layers of both images and Depth Maps, the method enhances the feature extraction capability of the network encoder. The integration of depth geometric information further strengthens the robustness of image segmentation. A simple decoding module is designed to decode the fused features, improving the inference speed of the RGB-D dual-stream semantic segmentation network while maintaining inference accuracy. Experiments show that RDFormer increases the FPS by 2 units on the Cityscapes dataset with an input resolution of 2048×1024, compared to the fastest existing RGB-D multimodal semantic segmentation algorithm, ESANet, while pre-serving corresponding inference accuracy.