Transformer-Based Visual Odometry from RGB-D information

This paper presents a novel RGB-D multimodal fusion visual odometry (VO) method which is designed for challenging environments, including noisy, low-light, and variable lighting conditions. Existing methods which rely solely on RGB images perform badly under challenging conditions. To address this issue, our approach firstly employs a two-branch encoder to extract multimodal features. Then, a multi-head self-attention mechanism is utilized within two-branched encoder for highlighting informative information. Thirdly, we incorporates a dense bundle adjustment (DBA) module to optimize pose and depth estimation. Experiments on the TUM and TartanAir datasets show that the proposed method outperforms existing VO methods, such as REVO, ORBVO and TartanVO, making it highly suitable for applications in robotics, augmented reality, and autonomous navigation.