A Hybrid Vison Transformer-Based Coarse-to-fine Remote sensing Image registration

In view of the fact that remote sensing image registration is easily affected by factors such as viewing angle and weather, resulting in low real-time performance and poor accuracy of registration results, this paper proposes an end-to-end remote sensing image registration algorithm based on hybrid visual transformers. A CNN-ViT hybrid architecture is proposed to quickly extract local features in the shallow layer of the network, and a partially asymmetric convolution APConv is proposed to reduce computational costs and improve feature sampling integrity. Through the combination of point-by-point convolution and group convolution, efficient interaction between local and global features is achieved. The self-attention mechanism is used in the deep layer to capture global information. The ELAFormer module is proposed to capture high- and low-frequency information through a dual-branch architecture, and the self-attention complexity is reduced by focusing linear attention. Then, a stacked linear cross-matching algorithm is used to obtain the coarse-level matching relationship, and then the coarse-level matching points and the fine-level map are combined. Cross-Attention is used to fuse fine-level features, and the matching matrix is optimized by combining the Sinkhorn algorithm. Finally, the affine transformation parameters are estimated through a deep regression network to achieve image registration. The experimental results of this paper show that the proposed method performs well in remote sensing image registration on complex terrain.