Fault-tolerant cross-modal pose realignment for heterogeneous air–ground localization
Air–ground localization becomes brittle when visual–inertial drift or an abrupt state jump places a camera pose outside the attraction basin of local camera–LiDAR registration. We address this recovery problem with a coarse-to-fine cross-modal pose-realignment framework. Visual features are back-projected and evaluated against local LiDAR planes through point-to-plane residuals, avoiding learned appearance correspondences at run time. An association-quality check determines whether the propagated pose is sufficiently supported by local geometry. When the support is insufficient, a bounded particle swarm optimization (PSO) search selects a coarse pose hypothesis, which is subsequently re-fined by a visual–inertial–LiDAR factor graph. The proposed framework therefore couples implicit geometric association, bounded basin exploration, and local multimodal refinement to recover from large pose errors. Experiments use the Eu-RoC sequences with a static Leica scan and real indoor and outdoor air–ground platforms with UGV-provided maps. In the reported severe cases, the method reduces translational RMSE by 33.8% under a 0.5 m initial offset and reduces post-disturbance mean absolute position error by 55.2% after a 30◦ mid-sequence yaw jump, relative to the best successful baseline in each test. Indoor tests with one UGV and three UAVs further demonstrate 20 Hz operation in a shared LiDAR map frame.
Authors
- Zhuozhu Jian (ORCID: https://orcid.org/0000-0003-2978-0875)
- Shengtao Zheng (ORCID: https://orcid.org/0009-0005-8438-3954)
- Bin Zhang
- Qixuan Li
- Xueqian Wang
- Xinlei Chen
- An Yi
- Lizhi Sun
- Ziyi Zhou
Institutions
- Dalian University of Technology (CN)
- University Town of Shenzhen (CN)
- Dalian University (CN)
- Tsinghua–Berkeley Shenzhen Institute (CN)
- Tsinghua University (CN)
Publication Details
- Journal
- Tsinghua Science & Technology
- Published
- 2026-11-20
- DOI
- https://doi.org/10.26599/tst.2026.9010106
- Primary Topic
- Robotics and Sensor-Based Localization
- Type
- article
- Field-Weighted Citation Impact
- 0.00