CMIF-VIO: A Novel Cross Modal Interaction Framework for Visual Inertial Odometry

Visual Inertial Odometry (VIO) estimates predicted trajectories through self motion. With the popularization of artificial intelligence, deep learning-based VIO methods have shown better performance than traditional geometry-based VIO methods. However, in deep learning methods, how to better achieve the fusion and complementarity between visual images from cameras and Inertial Measurement Unit (IMU) measurements from IMU sensors to output accurate pose remains a challenge. In this letter, we propose a novel Cross Modal Interaction Framework for VIO, named CMIF-VIO, which improves the accuracy of VIO and has good real-time performance. Specifically, we first used existing backbone network and built a simple backbone network to extract features from camera and IMU separately, ensuring low complexity. Then, we explored a cross modal interaction module that adaptively integrates information from different modal features, achieving deep interaction between visual and IMU modal features while maintaining feature dominance in each modal branch. Finally, a Long Short Term Memory (LSTM) network was introduced to model temporal motion correlation and output high-precision 6-degree-of-freedom (6-DOF) poses. The experimental results show that our method exhibits better performance compared to state-of-the-art VIO methods, and its real-time performance can meet the needs of practical application scenarios.