CaFI: Cascade Frustum-Image Fusion Network for 3D Object Detection

Recently, 3D object detection has seen a trend of employing representations under bird's eye view (BEV). However, most regions on the BEV map are empty, thus making the BEV-based detector computationally redundant, especially for detecting faraway objects. In contrast, the frustum-based detectors only focus on the regions of interest derived from the pre-detected 2D bounding boxes. These methods under-explore the texture information from the image since they only take the point clouds as inputs, leading to frequent failures in distinguishing objects with similar geometry features. To this end, a new frustum-based detector is revived with image information, namely the cascade frustum-image fusion network (CaFI). Based on the F-ConvNet method, this method extracts image and frustum features in parallel and then fuses them into a unified point representation. The fusion process is cascaded multiple times, and the fused features are fed into the detection head for the results. Experimental evaluation of the KITTI dataset showed that our model achieved better performance on both the test and validation sets.