Multiscale Pillars Fusion for 4-D Radar Object Detection With Radar Data Enhancement

The 4-D millimeter-wave (mmWave) radar has garnered significant interest in autonomous driving due to its robust ability to measure the range, azimuth, elevation, and velocity of targets. In addition, this is also attributed to its resilience in extreme environments, as well as lower costs. However, the sparsity and noise inherent in 4-D radar point clouds limit its perception of targets in practical application. To address this, we propose a radar point cloud enhancement (RPCE) method that leverages semantic knowledge from RGB images to enhance the quality of 4-D radar point cloud by generating virtual points and improving feature quality. Furthermore, we introduce MSPFNet, a multiscale pillars fusion network framework built on the enhanced point clouds. MSPFNet divides the point clouds into multiscale pillars and then extracts multiscale features from each pillar’s points. For these multiscale features, we propose a multiscale feature fusion (MSFF) module that can simultaneously capture intra- and interrelationship between different pillars through self-attention and cross-attention manners. Finally, we quantitatively and qualitatively evaluate our approach using the View-of-Delft (VoD) dataset and conduct extensive ablation studies to investigate its effectiveness. Our method’s performance is improved by 8.9% of mean average precision (mAP) in the entire annotated area and 5% of mAP in the driving corridor compared with the baseline. The experimental results and analysis indicate that the proposed method can effectively improve the performance for 4-D radar object detection with comparable inference time.