VPDNet: Virtual Point Density-Aware Network for Multimodal 3-D Object Detection

Lidar has become a prevalent sensor for 3D object detection in autonomous driving. However, the sparse and irregular nature of point cloud data obtained from Lidar necessitates the adoption of cross-modal object detection methods to enhance detection accuracy and stability. Nonetheless, the disparate representations of point clouds and images hinder comprehensive data fusion, leading to suboptimal performance. This paper proposes a novel multimodal density-aware 3D object detection method, VPDNet, and leverages depth completion-generated virtual point clouds to address the challenges of data fusion. To mitigate the interference caused by inaccurate depth completion, we introduce a virtual point cloud enhancement module. This module utilizes weighted features of virtual point clouds based on image segmentation information to suppress invalid information while retaining valid information. Furthermore, to further capture information from images and point clouds, we design an interactive attention fusion module that integrates features from virtual point clouds and real point clouds by adjusting attention weights. Additionally, the characteristics of Lidar dictate that collected point cloud data varies unevenly with distance, and point density, an essential feature, is often overlooked. Therefore, we propose a density-aware module that combines fused voxel features with kernel density estimation and point density information to extract spatial local features. Experiments conducted on the KITTI dataset demonstrate that our proposed 3D object detection algorithm improves pedestrian detection results by 6.93% compared to the baseline network. The code is available at https://github.com/Alexandriahui/VPDNet.