Hierarchical-Level Rain Image Generative Model Based on GAN

<div class="section abstract"><div class="htmlview paragraph">Visual perception systems for autonomous vehicles are exposed to a wide variety of complex weather conditions, among which rainfall is one of the weather conditions with high exposure. Therefore, it is necessary to construct a model that can efficiently generate a large number of images with different rainfall intensities to help test the visual perception system under rainfall conditions. However, the existing datasets either do not contain multilevel rainfall or are synthetic images. It is difficult to support the construction of the model. In this paper, the natural rainfall images of different rainfall intensities were first collected and produced a natural multilevel rain dataset. The dataset includes no rain and three levels (light, medium and heavy) of rainfall with the number of 629, 210, 248 and 193 respectively, totaling 1280 images. The dataset is open source and available online via: <a href="https://github.com/raydison/natural-multilevel-rain-dataset-NMRD" target="_blank">https://github.com/raydison/natural-multilevel-rain-dataset-NMRD</a>. Subsequently, a hierarchical-level rain image generative model, rain conditional CycleGAN (RCCycleGAN), is constructed. RCCycleGAN is based on the generative adversarial network (GAN), which can generate images of light, medium and heavy rain by inputting no rain images into the model. In the process of model tuning, a total of three modifications are made based on the DerainCycleGAN, including introducing different rainfall intensity labels, replacing the activation function, and adjusting the training strategy. Compared with the two baseline models, CycleGAN and DerainCycleGAN, the peak signal-to-noise ratio (PSNR) of RCCycleGAN on the test dataset is improved by 2.58 dB and 0.74 dB, and the structural similarity (SSIM) is improved by 18% and 8%, respectively. The ablation experiments are also carried out and validate the effectiveness of the model tuning.</div></div>