SG-RoadSeg+: End-to-End Freespace Detection Upgraded at Data, Feature, and Loss Levels

Freespace detection is essential for autonomous robotic systems, particularly self-driving vehicles that depend on precise environmental perception for safe navigation. State-of-the-art methods typically use dual encoders to extract features from RGB images and other data sources, which are then fused and decoded for freespace prediction. This article presents SG-RoadSeg+, which incorporates the transformed disparity-azimuth-zenith (TAZ) map and a novel freespace disparity projection linearity (FDPL) loss function. The key advancement is the TAZ map, which integrates transformed disparity and surface normal maps to enrich spatial understanding, necessary for accurate freespace detection. Additionally, the FDPL loss function optimizes the linear projection of the disparity during training, ensuring precise environmental measurements. Moreover, our data augmentation technique, using random perspective transformations, primarily enhances the system’s generalization to real-world environments, enabling it to handle complex and dynamic scenarios more effectively. Our previous model, SG-RoadSeg, uses a deep stereo encoder, resulting in strong freespace detection performance. However, its reliance on a single estimated disparity map limits the available spatial information, potentially leading to inaccuracies. Comprehensive experiments on the KITTI Road and Semantics datasets show that SG-RoadSeg+ outperforms the previous version, with improvements of up to 0.73% in IoU and 0.51% in F-score. Notably, applying the TAZ map to other feature-fusion models improves F-score by 3.27% and IoU by 6.44%, highlighting its effectiveness in boosting freespace detection accuracy.