TSAM: multilevel condition-enhanced image inpainting via progressive feature fusion and decoding mechanism

In recent years, deep learning-based image inpainting methods have made significant progress by incorporating structural priors. Due to the lack of information interaction between textures and structures and the difficulty of integration, these methods are inadequate in handling complex inpainting tasks, which lead to structural distortion and detail loss. This paper proposes a novel twin-stream adversarial model (TSAM) for image inpainting. The model establishes a Multilevel Conditional Encoding (MCE) network in the twin-stream encoding stage, which enriches the information interaction between textures and structures. The MCE network incorporates both the structure-constrained texture synthesis and texture-guided structure reconstruction, which facilitates mutual improvement and improves the effect. In the twin-stream decoding stage, a Progressive Feature Fusion and Decoding (PFFD) network is constructed to fuse textures and structures, guaranteeing the global consistency and rich details of inpainting results on complex scenes. Qualitative and quantitative experiments on the CelebA-HQ, Paris StreetView, and Places2 datasets show the superiority of this method.