Feature Level Fusion of Radar and Camera based on Transformer in 3D Object Detection

In the field of 3D object perception, millimeter-wave radar and cameras have complementary information. However, the sparseness of millimeter-wave radar point clouds and the differences in camera and radar data make it difficult to fuse the features of the two sensors. In order to achieve further correlation between radar and cameras, much research has been invested in camera-radar fusion 3D object detection. We propose a new radar point cloud processing network and a feature-level fusion method based on Tranformer, which uses radar point to supplement camera information at a fast inference speed to achieve effective radar camera fusion. The input of the radar network does not need to exclude invalid radar points in advance, and the unstructured and disordered radar point cloud after clustering is directly input. The output not only includes the information carried by the radar point, but also obtains the structural information between radar groups. The fusion network obtains refined detection results through interactive fusion of features. The entire model structure is simple and no manual methods are used in network inference. Compared with the single-view method, this method improved by 5.3% on NDS, and the errors in orientation, translation, velocity and attributes were reduced.