CycFormer: Unsupervised Rain Removal Network Based on CycleGAN and Transformer

Rainy weather presents significant challenges for applications relying on visual perception in intelligent transportation systems. The scarcity of real paired training data complicates single-image rain removal tasks, prompting an increasing interest in unsupervised methods capable of handling real-world rainy images without paired data. At present, most unsupervised rain removal methods are based on the CycleGAN framework; however, the combination of this framework and transformer is not satisfactory owing to most Transformers’ insufficient ability to model real rain features with global inhomogeneous distributions, which prevents them from being fully applicable to unsupervised tasks. This study devised an unsupervised rain removal network based on CycleGAN and the DerainFormer transformer. First, a deformable sparse attention mechanism was developed to improve the Transformer’s suitability for unsupervised tasks in CycleGAN architectures. Subsequently, a two-stage alternating transformer structure was designed to enhance its global non-uniform modeling capabilities for real rain images, In addition, a dual-channel parallel feed-forward network was used to establish the correlation between multiscale rain stripes. Finally, since rain removal is considered a decomposition task, a rain layer unsupervised training method for joint positional contrastive learning was proposed to separate the rain streaks effectively. We conducted several experiments on different real and synthetic rain datasets and the results confirmed that our unsupervised rain removal method performed well. The source code will be released at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/derainsipl/CycFormer</uri>.