Hybrid Sparse Transformer and Feature Alignment for Efficient Image Completion

Abstract In this paper, we propose an efficient single‐stage hybrid architecture for image completion. Existing transformer‐based image completion methods often struggle with accurate content restoration, largely due to their ineffective modeling of corrupted channel information and the attention noise introduced by softmax‐based mechanisms, which results in blurry textures and distorted structures. Additionally, these methods frequently fail to maintain texture consistency, either relying on imprecise mask sampling or incurring substantial computational costs from complex similarity calculations. To address these limitations, we present two key contributions: a Hybrid Sparse Self‐Attention (HSA) module and a Feature Alignment Module (FAM). The HSA module enhances structural recovery by decoupling spatial and channel attention with sparse activation, while the FAM enforces texture consistency by aligning encoder and decoder features via a mask‐free, energy‐gated mechanism without additional inference cost. Our method achieves state‐of‐the‐art image completion results with the fastest inference speed among single‐stage networks, as measured by PSNR, SSIM, FID, and LPIPS on CelebA‐HQ, Places2, and Paris datasets.