RNOSMamba: Boosting Road Negative Obstacles Segmentation via Vision Mamba from RGB and Depth Images

The fusion of RGB and depth information holds significant potential for accurate road negative obstacle identification. However, effectively leveraging these multimodal data for distinguishing fine-grained road surface defects, such as potholes and cracks, remains a challenge. Inspired by the recent progress of multimodal fusion in a variety of computer vision tasks, this paper aims to propose a novel Vision Mamba based Road Negative Obstacle Segmentation framework (RNOSMamba) that leverages the complementary strengths of optical and depth images. Toward this end, optical and depth images in the feature domain are appropriately fused to boost the performance of road negative obstacle segmentation (RNOS). The hierarchical decoder incorporates Cross-Modality State Space (CMSS) blocks and Cross-Scale Feature Fusion (CSFF) modules to refine features and produce precise segmentation masks. Extensive experiments demonstrate that the proposed RNOMamba was able to achieve 68.5% mIoU and 80.7% mF1, highlighting its potential to significantly boost the accuracy of road negative obstacle segmentation.