Multi-Modal Geometric Learning for Robot Localization

Robot localization uses external sensor data to estimate a robot's position, but movements can alter vision measurements and learned features, impacting robustness. This thesis introduces a new perspective on multi-modal robot localization by integrating geometric learning, primarily focusing on equivariant networks to address these challenges. First, we explore the application of geometric learning in the loop closure process, enabling the robust identification of previously visited locations despite transformative changes. This framework's SE(3)-invariant attribute offers stable, consistent descriptors regardless of changes in robot pose. Next, we propose SE(3)-equivariant transformer designs for learning point cloud correspondences, crucial for point cloud registration tasks. Through adopting equivariant feature learning, the framework significantly improves the robustness of point cloud registration with low overlap and significant pose changes. Finally, we integrate geometric learning with visual foundation models for localization tasks. The proposed fusing methods bridge the gap that combining images with equivariant point cloud features while preserving their equivariance is challenging due to the group structure that is added to the network architecture. Overall, this dissertation explores the benefits of incorporating geometric constraints into machine learning models, significantly enhancing both the reliability and performance of robot localization and contributing to critical advancements in robotics.