3D Pedestrian Detection Based on Virtual LiDAR Points and Adaptive Residual Fusion

Recent multi-modal object detection methods have achieved impressive results by leveraging virtual point clouds generated from images. These virtual points enrich object features, facilitating more accurate bounding box regression and classification, which in turn enhances detection accuracy. This is crucial for small objects such as pedestrians, which typically have sparser raw points, making feature extraction more challenging. However, when images are unavailable, these models will have a significant performance degradation or even not work. Motivated by the image-based virtual point method, we introduce a straightforward approach to generate virtual LiDAR points via mask average pooling. Additionally, we propose a LiDAR-based object detection model utilizing these virtual points. We incorporate a 3D adaptive residual fusion module with dilated convolution in backbone to extract voxel features more effectively. Furthermore, a 2D adaptive residual fusion module is used to aggregate BEV features derived from raw and virtual points to obtain more accurate pedestrian proposals. To facilitate the fusion of BEV features, we employ an auxiliary heatmap loss for additional supervision. Our model achieves an mAP of 81.41% on KITTI dataset, with an AP of 72.79% for pedestrians, demonstrating the effectiveness of our approach. Additionally, we validate that our virtual points generation method can mitigate the performance degradation observed in existing multi-modal virtual point models when images are unavailable.