Dynamic Multi-Scale Heterogeneous Feature Fusion for RGB-X Freespace Detection

Freespace detection is an essential element of autonomous driving perception systems. The extraction of features from RGB images and an additional source or modalities of positional information, like depth images, followed by constructing a feature fusion network utilizing a duplex encoder, has become a dominant paradigm in this field. State-of-the-art approaches typically employ this architectural configuration. However, the substantial computational burden associated with this approach has a negative impact on real-time performance. Moreover, more research is needed to investigate the potential of the feature fusion method. In this paper, we seek to address these limitations by building on our previous work, SNE-RoadSeg. To overcome these limitations, we present a new RGB-X feature fusion network architecture, the Heterogeneous Dynamic Decoding Former (HDD-Former). It uses a weight-sharing multi-scale encoder to extract features fully fused with our proposed new heterogeneous feature fusion module, the Residual Sliding Feature Fusion Module. It enables the efficient fusion of global and local features. A multi-scale decoder is then used to generate the freespace prediction. The experimental results on VKITTI, KITTI road, and KITTI semantics datasets demonstrate that our proposed HDD-Former validates the effectiveness of our proposed architecture in achieving the best overall performance among their datasets.