Sef-VIO: Self-attention-based Multimodal Fusion for Neural Visual-Inertial Odometry

Learning-based visual-inertial odometry (VIO) has been attracting much attention for its significance in robot navigation. However, most existing methods fail to accurately estimate the robot’s trajectory via visual and inertial information, primarily due to two key shortcomings: 1) treat all different dimensions of inertial information equally neglecting inherent disparities within inertial data; 2) lack effective fusion strategies for integrating visual and inertial data. To address the above limitations, we propose a novel Self-attention-based Multimodal Fusion (Sef-VIO) method, designed to fuse visual and inertial data for precise robot trajectory estimation. More specifically, we introduce a channel-aware inertial feature extraction method to capture inherent disparities within inertial data effectively. Additionally, we present a visual-inertial feature fusion method that is based on self-attention mechanisms, which can effectively leverage both global and local context information to compute the correlation between visual and inertial modalities accurately. Extensive experiments are carried out on benchmark datasets, such as the KITTI Odometry dataset, to demonstrate the effectiveness of our proposed Sef-VIO method.