Accurate and Robust Roadside 3-D Object Detection Based on Height-Aware Scene Reconstruction

Roadside 3-D object detection allows for a drastic expansion of the visibility range and a reduction in occlusions for autonomous vehicles. Recent approaches are based on bird’s-eye view (BEV) fusion, which unifies multimodal features in the shared BEV representation space. However, the camera-to-BEV projection throws away the geometric information of camera features, hindering the effectiveness of such methods. Besides, depth-based camera lifting results in inefficiency and instability in disrupted roadside scenarios. To address these challenges, this article introduces a novel 3-D object detection framework based on height-aware scene reconstruction, dubbed HSRDet. Specifically, we leverage height-aware 3-D reconstruction to ensure geometric consistency in BEV feature mapping and employ a fast camera-to-BEV transformation based on feature distillation to boost efficiency without compromising performance. In addition, we integrate a novel data augmentation method, namely, View Shake (VS), to further improve the performance of our model. Extensive experiments on the DAIR-V2X dataset demonstrate that HSRDet not only achieves state-of-the-art detection accuracy but also exhibits strong robustness in disturbance scenarios. Further experiments on the intelligent roadside units (RSUs) have revealed that our method runs stably at 11.8 frames/s on a RTX 3090 Ti GPU, thus promising vast engineering application prospects.