3D Vehicle Detection From Roadside Monocular View Based on Improved CenterNet

At present, automatic driving mainly obtains vehicle 3D perception information through multi-sensor fusion but only makes part use of monocular traffic surveillance cameras widely deployed on the roadside. Therefore, a 3D vehicle detection method based on CenterNet is proposed in this paper, which can estimate the 3D position and attitude of vehicle targets from a roadside monocular view. <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup>Firstly, the camera with a general roadside monocular view angle is calibrated to obtain the projection matrix from 3D to 2D, and the ground plane equation in the camera coordinate system. Secondly, the CenterNet deep network with a feature fusion module is used as the basis. A network module for accelerating convergence is added, and an optimization loss function is designed according to the prior geometric constraints of vehicles to improve the detection effect of the network for different scales and types of vehicles; finally, the output results of the network are decoded, and the three-dimensional shape of vehicles is calculated according to the spatial geometric constraints of vehicles. The experiments in this paper are performed on the public dataset Rope3D. For vehicle prediction results, AP| R40 reaches 61.60%. The final evaluation of the vehicle's 3D shape is carried out. Compared with the existing network model, the final results are improved by 2.18 and 6.75 percentage points, respectively, when the 3D frame-to-frame ratio is 0.5 and 0.7. In addition, the impact of the network improvement module on the accuracy of the final results is verified by multiple ablation experiments. The final test effect of the model reaches 47FPS on the Rope3D dataset, which meets the real-time requirements.