Low-Cost Fusion Odometry Algorithm Based on 4D Radar and Pseudo-LiDAR: Bridging the Gap between 4D Radar and Images in 3D Space

<div class="section abstract"><div class="htmlview paragraph">Recently, four-dimensional (4D) radar has shown unique advantages in the field of odometry estimation due to its low cost, all-weather use, and dynamic and static recognition. These features complement the performance of monocular cameras, which provide rich information but are easily affected by lighting. However, the construction of deep radar visual odometry faces the following challenges: (1) the 4D radar point cloud is very sparse; (2) due to the penetration ability of 4D radar, it will produce mismatches with pixels when projected onto the image plane. In order to enrich the point cloud information and improve the accuracy of modal correspondence, this paper proposes a low-cost fusion odometry method based on 4D radar and pseudo-LiDAR, 4DRPLO-Net. This method proposes a new framework that uses 4D radar points and pseudo-LiDAR points generated by images to construct odometry, bridging the gap between 4D radar and images in three-dimensional (3D) space. Specifically, the pseudo-LiDAR point cloud is obtained by back-projecting the depth map generated by the image into 3D space, which changes the way the image is represented and effectively alleviates the problem of sparse 4D radar point cloud. In order to fully integrate the two modalities, we designed a cross-attention-based grouped fusion module, which uses the accurate spatial measurement of radar to restore the scale of pseudo-LiDAR points, and groups based on ball queries in the metric spatial scale to achieve accurate association and bidirectional fusion of features. Finally, we conducted experiments on the View-of-Delft (VOD) dataset and compared and verified the excellence of the method and the effectiveness of each module.</div></div>