Spatial Semantic Fusion Network for Autonomous Driving Visual Joint Perception Algorithm

[Purposes] The visual joint perception system can fulfill multi-tasks such as traffic object detection, drivable area segmentation, and lane detection in autonomous driving traffic scenes, which is essential in autonomous driving. In practical application, the accuracy and speed should be appropriately balanced. The autonomous driving visual joint perception network YOLOP has achieved great performance in real-time. However, it ignores the feature conflicts of different scales in the feature pyramid network and the texture details loss in the downsampling process. To relieve these problems, the spatial semantic fusion network for autonomous driving visual joint perception (SSFJP) algorithm is proposed. In this paper, the original semantic fusion network of YOLOP is modified from two aspects, focusing on spatial semantic embedding and fusion. [Methods] Regarding feature enhancement, the bidirectional attention information strength module (BAISM) was adopted for global contextual prior and corresponding precise positional information modeling from horizontal and vertical dimensions, which helps embed channel attention semantic information into spatial details, effectively highlight the critical visual area, and improve the representation ability of features' texture details. In terms of feature fusion, the multi-branch cascade feature fusion (MCFF) used atrous convolution with different rates and exponentially weighted pooling to fuse scene feature information of different scales, cascade fusion of spatial context semantic information, relieve the mutual interference of features corresponding to spatial positions of different levels, and achieve complementary information between texture details and high-level semantics. Besides, adaptive parameters were introduced to design weighting coefficients of the loss function to solve the imbalanced training of different sub-tasks, effectively improving the detection and segmentation performance. [Findings] Experiments on the BDD100K dataset show that the proposed autonomous multi-task driving joint perception, model SSFJP, compared with YOLOP, guarantees real-time detection and increases the average accuracy of lane line detection and object detection by 8.9% and 1.6%, respectively.