Diffusion Models for Cross-Domain Image-to-Image Translation with Paired and Partially Paired Datasets

The line-art colorization problem is a task in generative modeling with the goal of generating colored artworks from an artist's hand-drawn line-arts. Machine learning models such as generative adversarial networks have been applied to this task. At the time of this research, the application of diffusion models to this task has not been well studied, despite the impressive results in image generation that diffusion models have demonstrated. We propose to apply conditional diffusion models to the line-art colorization problem and expand the capability of such models by proposing conditional cross-domain diffusion models, capable of a two-way transformation between image domains. The main findings are 1) the conditional diffusion models are effective at the task of line-art colorization and they provide state-of-the-art results compared to previous methods, and 2) the proposed conditional cross-domain diffusion models are capable of two-way cross domain image-to-image translation with high quality results and they can be trained on both paired and partially paired images.<sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup>1An implementation is provided at https://github.com/aaroswings/orthogonal_cross_domain_diffusion.