SPWS‐Transformer: A Study of 3D Target Detection Method Based on Lightweight Depth Prediction With Multi‐Scale Fusion

ABSTRACT Advanced driver assistance systems (ADAS) mainly consist of three components: environmental perception, decision planning, and motion control. As a fundamental component of the ADAS environmental perception system, 3D object detection enables vehicles to avoid obstacles and ensure driving safety only through accurate and real‐time prediction and localization of three‐dimensional targets such as vehicles and pedestrians in road scenes. Therefore, to improve both the real‐time performance and accuracy of 3D object detection, we propose a lightweight depth prediction‐based 3D object detection model with multi‐scale fusion—SPWS‐Transformer. First, to enhance the model's accuracy, we propose a feature extraction network incorporating multi‐scale feature fusion and depth prediction. By designing a multi‐scale feature fusion module, we effectively combine multi‐scale semantic and fine‐grained information from feature maps of different scales to enhance the network's feature extraction capability. To capture spatial information from the feature maps, we apply convolution, group normalization, and nonlinear activation operations on the fused feature maps to generate depth feature maps. Both the fused feature maps and depth feature maps serve as inputs for subsequent network stages. To further improve accuracy, we leverage the long‐range modelling advantages of Transformers by designing a feature enhancement encoder to strengthen the representation capability of depth feature maps. We incorporate a dilated encoder to perform positional encoding on depth feature maps and utilize multi‐head self‐attention mechanisms to capture contextual relationships within the input scene, thereby enhancing the detection capability of the 3D object detection network. Then, to improve real‐time performance, we design a decoder structure with scale‐aware attention. By predefining masks of different scales, we adaptively learn a scale‐aware filter using depth and visual features to enhance object queries. Finally, on the KITTI dataset, the improved algorithm achieves an AP of 24.66% for the car category, with more significant improvements in detection accuracy under the ‘hard’ difficulty level. The model achieves an inference time of 24 ms.