MonoSG: Monocular 3D Object Detection With Stereo Guidance

In the context of autonomous driving, monocular 3D detection is regarded as a fundamental and essential task due to its convenience, speed, and low cost. However, the lack of depth information in monocular images presents significant challenges for predicting object 3D information. Although existing methods address this issue using LiDAR guidance or pre-trained depth estimators, their substantial computational resource requirements limit scalability in real-world applications. In this letter, we propose a novel monocular 3D object detector with stereo guidance, called MonoSG. It simulates human visual perception by using stereo images during training to guide learning and retrieves the right-view and depth information from monocular images during inference for accurate 3D detection. The Stereo Guidance Cross Attention Module (SG-CAM) is designed to fuse binocular image information. Intra-view features are extracted from binocular images, and cross-attention is computed from the left to the right view. Then, the cross-attention features are fused with the intra-view features of the left view, enabling stereo guidance for MonoSG. To better adapt to different data distributions and improve the generalization ability of MonoSG, the Stereo Guidance Auxiliary Labels (SG-AL) are introduced for each object of stereo images, with DIoU3D proposed as the label score. Furthermore, the SG-AL score loss is proposed to guide the MonoSG that can reduce gradient variance, facilitate network convergence, and mitigate the issue of insufficient depth information. Comprehensive experiments on the KITTI dataset validate the effectiveness of our method, demonstrating superior performance, particularly on low-resolution images.