Unsupervised Deep Image Stitching for Remote Sensing: Aligning Features Across Large-Scale Geometric Distortions

Remote-sensing images frequently exhibit complex and extensive geometric distortions due to the characteristics of imaging platforms and conditions. These distortions not only increase the errors in feature extraction and matching of traditional stitching methods but also hinder the learning process of the model in recent deep stitching solutions. To address these problems, we propose an unsupervised deep stitching pipeline for remote-sensing images (UDRSIS). Firstly, it includes a progressive alignment procedure that comprises coarse-grained alignment (CGA) and fine-grained alignment (FGA) for accurate and robust registration. A Homography Transformer (HomoFormer) architecture is devised to provide a rigid and fundamental matching of regions in CGA based on contrast and non-local features. Subsequently, a Thin-Plate Spline Transformer (TPSFormer) is developed to ensure flexible shape preservation in FGA. Additionally, we endow our HomoFormer with SIFT-Guided Curriculum Learning (SGCL) to boost the alignment's ability to handle distortions. Finally, a Seam Transformer (SeamFormer) is designed to seamlessly composite the stitched image using omnidirectional composition masks. Given the absence of an evaluation benchmark, we construct a comprehensive dataset, namely Unsupervised Deep Stitching of Aerial Images Dataset (UDAIS-D). The simulated data within it encompasses a wide range of geometric distortions and radiometric distortions as well as other noises are also considered. The real-world data is collected from actual remote-sensing imaging results. Considering the challenge of evaluating the stitching quality, we present Large Visual Language Models (LVLMs)-based metrics to leverage their powerful capabilities for the evaluation of remote-sensing image mosaicking. Numerous experiments demonstrate that our method surpasses other state-of-the-art solutions. The codes and data will be available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/yyywxk/UDRSIS</uri>.