SeFENet: Robust Deep Homography Estimation via Semantic-Driven Feature Enhancement

Images captured in harsh environments often exhibit blurred details, reduced contrast, and color distortion, which hinder feature detection and matching, thereby affecting the accuracy and robustness of homography estimation. While visual enhancement can improve contrast and clarity, it may introduce visual-tolerant artifacts that obscure the structural integrity of images. Considering the resilience of semantic information against environmental interference, we propose a semantic-driven feature enhancement network for robust homography estimation, dubbed SeFENet. Concretely, in our homography estimation network —— Target Aware Homography Estimation Module(TAHEM), we first introduce an innovative hierarchical scale-aware module to expand the receptive field by aggregating multi-scale information, thereby effectively extracting image’s structural features under diverse harsh conditions. Subsequently, we employ a Semantic Extraction Module to extract multi-scale semantic features from the input images. Combined with a high-level perceptual framework, this enables degradation-tolerant semantic feature extraction. Building upon this, the Semantic-Guide Meta Constraints module leverages a meta-learning training strategy to effectively fuse the semantic features with structural features. By internal-external alternating optimization, the proposed network achieves implicit semantic-wise feature enhancement, thereby improving the robustness of homography estimation in adverse environments by strengthening the local feature comprehension and context information extraction. Experimental results under both normal and harsh conditions demonstrate that SeFENet significantly outperforms SOTA methods, reducing point match error by at least 41% on the large-scale datasets.