Multi-Modal 3D Object Detector with Object-Guided Fusion and Hierarchical Sample Selection

Accurately detecting objects in 3D scenes is crucial for autonomous driving. Although existing voxel-based methods have achieved remarkable progress, their performance on tail objects remains unsatisfactory. We identify two core issues contributing to this phenomenon: the detectors frequently misidentify some background elements as foreground objects, and there is a misalignment between the classification score and detection quality. To tackle these challenges, we introduce an object-level – guided multi-modal 3D object detector with an object-guided feature fusion (OFF) module and a hierarchical sample selection (HSS) strategy, named OGMMDet. Specifically, OFF introduces rich image features to enhance the representation of objects while using an object distribution heatmap to suppress the background. This approach provides geometry clues for tail objects while providing category priors to filter out the background. HSS uses a local-to-global ranking approach to calculate the relative classification loss weights of all proposals. It assigns higher weights to proposals with higher IoU when optimizing classification branches. This ensures that the model focuses its optimization on these higher-quality proposals. Consequently, there is a positive correlation between the classification score and IoU. This method alleviates the misalignment between the classification score and detection quality. Extensive experiments on the KITTI and nuScenes benchmarks demonstrate the effectiveness of our OGMMDet, which achieves 45.61% and 68.96% mean average precision (mAP) on pedestrians and cyclists on the KITTI benchmark, respectively. Code is available at https://github.com/ZhongJianPing1/ogmmdet.git .