Deep Depth‐Flow Odometry With Inertial Sensor Fusion

ABSTRACT Robust pose estimation in GNSS‐denied environments is essential for autonomous driving systems. Recent advancements in deep learning within the field of computer vision have significantly contributed to the development of visual odometry (VO). However, most VO‐based approaches still suffer from scale drift errors in general road environments. This letter introduces a novel visual‐inertial odometry framework for robust pose estimation. Inertial measurement unit (IMU) measurements obtained between image frames are fused with deep learning‐based optical flow and depth predictions extracted from image pairs. First, measurements from the accelerometer and gyroscope are propagated through the IMU dynamic model. Next, optical flow and depth information predicted from image pairs are used in a geometric approach to recover the camera motion by optimising correspondences based on optical flow consistency. Finally, the proposed method is evaluated on the publicly available KITTI dataset, and its performance is compared with existing methods. Additionally, the impact of the network model on flow consistency, which plays a crucial role in geometry‐based pose recovery, is analysed. The results demonstrate that the proposed method achieves reliable pose estimation accuracy.