An Integration Framework for the Inpainting and Colorization of Arbitrary Masked Grayscale Images

Image inpainting and colorization are crucial tasks in image restoration, yet existing methods typically address them independently, limiting their applicability to complex real-world scenarios. Conventional inpainting techniques struggle with maintaining structural consistency in arbitrarily masked regions, while colorization models often assume fully restored grayscale inputs. To bridge this gap, we propose a unified framework that seamlessly integrates inpainting and colorization into a single pipeline. Our approach leverages three state-of-the-art models: the segment anything model (SAM) for detecting arbitrary masked regions, the mask-aware transformer (MAT) for structurally coherent inpainting, and the dual-decoder colorization model (DDColor) for producing perceptually consistent colorized outputs. By integrating these models, our framework effectively restores both structure and color, handling arbitrary masks with greater robustness than existing methods. Experimental results on diverse datasets demonstrate superior performance in both qualitative and quantitative evaluations. Our findings highlight the advantages of a unified approach to image restoration and provide a foundation for future advancements in multi-stage deep learning frameworks.