Color and Attention for U : Modified Multi Attention U-Net for a Better Image Colorization

Image colorization is a tedious task that requires creativity and understanding of the image context and semantic information. Many models have been made by harnessing various deep learning architectures to learn the plausible colorization. With the rapid discovery of new architecture and image generation techniques, more powerful options can be explored and improved for image colorization tasks. This research explores a new architecture to colorize an image by using pre-trained embeddings on U-Net combined with several attention modules across the model. Using embeddings from a pre-trained classifier provides a high-level feature extraction from the image. Conversely, multi-attention gives a little taste of image segmentation so that the model can distinguish objects in the image and further enhance the additional information given by the pre-trained embeddings. Adversarial training is also utilized as a normalization to make the generated image more realistic. This research preferred Parch GAN over base GAN as the discriminator model to ensure that the colorization has a consistent quality across all patches. The study shows that this U-Net modification can improve the generated image quality compared to a normal U-Net. The proposed architecture reaches an FID of 48.6253, SSIM of 0.8568, and PSNR of 19.7831 by only training it for 25 epochs; hence, this research offers another view of image colorization by using modules that are often used for image segmentation tasks.