MC-Refine: Enhanced Cross-Modal 3-D Object Detection via Multistage Cross-Scale Fusion and Box Refinement

End-to-end DETR-based cross-modal fusion in 3-D object detection has achieved promising performance in many benchmarks. However, these methods either implement cross-modal fusion in a single stage using coarse voxel features, overlooking the benefits of multistage fusion and the detail loss due to decreased feature resolution, or merely encode voxel center coordinates as queries to consider the entire global spatial context. Consequently, performance is limited. In this article, we propose MC-Refine, an end-to-end DETR-based cross-modal 3-D object detection framework designed for fusing features from both the global scene and local object region. We first design multistage voxel feature fusion (MVF), introducing point-to-voxel mapping, to boost deep fusion of the dense image feature and sparse point cloud feature across voxelization and 3-D backbone stages. In addition, a cross-scale gated-attention graph (CGG) is proposed to construct a cross-scale voxel graph, enabling the capture of detailed spatial information from high-resolution (HR) voxel spaces. For the local object region, we propose a refinement stage to finetune the object location and scale, which aggregates various types of powerful features to improve the accuracy of the entire detection pipeline. Extensive experiments on the nuScenes and KITTI 3-D object detection benchmarks show that the proposed MC-Refine consistently yields performance enhancements. Notably, our proposed modal can achieve better performance compared with the state-of-the-art (SOTA) algorithms on the nuScenes dataset.