An Efficient Transformer Incorporating a Spatial Decay Matrix and Attention Decomposition for Image Matching

Homography estimation has widespread applications in visual computing and image analysis. Current state-of-the-art Transformer-based image matching methods require computing all possible key-value combinations to obtain global information, resulting in high computational costs and slow inference speeds. This paper presents an enhanced Transformer-based approach for image matching, integrating a spatial decay matrix derived from Manhattan distance and leveraging attention decomposition. We introduce Manhattan Self-Attention (MaSA) to enhance the LoFTR framework. This approach maintains the integrity of the spatial decay matrix while reducing the computational burden of modeling global information. Through the incorporation of a spatial decay matrix and attention decomposition, explicit spatial priors can be seamlessly integrated into the visual backbone network, enabling the achievement of linear computational complexity. Thus, it improves decoding throughput, latency, and GPU memory usage without sacrificing performance. Experimental evaluations indicate that the proposed method substantially enhances matching accuracy while effectively reducing inference time and memory overhead, outperforming state-of-the-art approaches under consistent evaluation metrics.