NIR-to-RGB image colorization based on a conditional GAN

Near-infrared (NIR) image colorization is an image enhancement method that improves the readability of a near-infrared image and enhances its semantic information. For the problems of color distortion, semantic ambiguity, and unclear texture shape in current near-infrared image colorization techniques, we propose a novel, to the best of our knowledge, near-infrared image colorization approach based on the generative adversarial network (GAN). This method carries out an optimization design from the generator and discriminator of the GAN and designs a proper loss function based on the new network architecture. In terms of the generator, we design and integrate a Res-WTConv-U-Net network within the generator. Additionally, we design a deep bottleneck block, which is composed of a residual block and an efficient attention module (ECA), to replace the bottleneck layer in the U-Net. Moreover, we replace the traditional convolution with the wavelet convolution (WTconv) to achieve more effective feature extraction and better performance. In the aspect of the discriminator, we design a dual-scale discriminator by combining two discriminators of different receptive fields, which can take into account the global structure and local details. After training the model using the same dataset, we conduct a comparative experiment against typical image colorization methods. In this experiment, structural similarity (SSIM), peak signal-to-noise ratio (PSNR), and color histogram similarity (CHS) are employed as image evaluation indices. By comparing the colorization results in two different datasets, it is concluded that the PSNR has improved by 12.6% on average, the SSIM has improved by 7.4% on average, and the CHS has improved by 9.5% on average. Experimental results show that the colorization effect of this method is significantly better than other methods.