Height3D: A Roadside Visual Framework Based on Height Prediction in Real 3-D Space

In recent years, vision-based roadside 3D object detection has received a great deal of attention, which is an important part of the Intelligent Transportation System (ITS). It extends the perception range beyond the limitations of Autonomous Vehicle (AV) and enhances road safety. While previous work mainly focuses on height prediction in image 2D space, which is limited by the perspective property of near-large and far-small on images, making it difficult for network to understand real dimension of targets in the 3D world. Inspired by this insight, a roadside visual framework Height3D based on height prediction in real 3D space, is proposed. Height Prediction Block (HPB) with explicit height supervision is proposed in real 3D space instead of in image 2D space to predict the height distribution of targets for roadside view transform. Also, Spatial Aware Block (SAB) is used to further extracts spatial context information in BEV space and enhances fine-grained BEV features. The proposed method is applied to two large-scale roadside benchmarks, DAIR-V2X-I and Rope3D. Extensive experiments are performed to verify its effectiveness. The proposed Height3D outperforms the state-of-the-art methods of (1.15, 7.37, 4.03) Average Precision (AP) for Vehicle, Pedestrian and Cyclist categories in 3D object detection task, respectively. Meanwhile, the proposed method achieves 31.55 FPS without using any CUDA or TensorRT acceleration. The code is available at https://github.com/zhangzhang2024/Height3D