RTO-LLI: Robust Real-Time Image Orientation Method With Rapid Multilevel Matching and Third-Times Optimizations for Low-Overlap Large-Format UAV Images

UAV real-time photogrammetry is important to promote the rapid generation of photogrammetry 4D product, intelligent information extraction and rapid remote sensing mapping, and efficient large-scale 3D modeling. However, for real-time processing of low-overlap large-format image sequence, there remains two challenges: (1) Large-format images result in greater data volume and computational load, posing challenges for real-time online processing on regular-performance computing units, requiring more efficient algorithms; (2) Low-overlap images make it difficult for matching correspondences to cover the entire overlapping area at real-time, leading to significant challenges for real-time and robust relative orientation. Therefore, this paper proposes a robust Real-Time Orientation method for Low-overlap Large-format UAV Images (RTO-LLI), which can robustly handle these kind of data in real-time. Firstly, robust initialization method for real-time processing of low-overlap large-format images was designed to ensure a high-success-rate of SLAM initialization. Secondly, constant velocity hypothesis tracking enables fast orientation during constant-speed flight. Thirdly, when the second step false, using real-time pose estimation method based on multilevel matching and coarse-to-fine optimization to robustly solve the precision image pose. Fourthly, final (third-level) pose optimization method based on the IRLS algorithm with suitable search area, which can compute higher-precision image pose in real-time. Finally, real-time mapping based on parallel processing for low-overlap images can generate high-precision 3D point maps and complete feature extraction for the next frame in real-time. Experiments conducted on several different types of scenes show that: (1) the processing speed of RTO-LLI significantly surpasses traditional offline methods: PhotoScan, OpenMVG, Colmap. RTO-LLI can handle large-format UAV image sequence (single-imagery has 20-million-pixels) at a speed of 1.5 frames-per-second, meeting the demands of real-time UAV photogrammetry tasks; (2) RTO-LLI is the only method that has successfully completed real-time tasks in all 50-times repeated experiments for four different types of scenes, demonstrating robustness far superior to other classical SLAM solutions; (3) the-displacement-error of the estimated Pose by RTO-LLI is less than 1/2000 of the-trajectory-length, and the average-reprojection-error is less than 1.5 pixels, almost as well as traditional offline methods. RTO-LLI method meets the efficiency, robustness and accuracy requirements of real-time photogrammetry for low-overlap large-format UAV images.