CUT-P2PGM: Task-Aware Unpaired Mid-Wave Infrared Image-to-Image Translation via Pseudo-paired Generative Modeling

Abstract Unpaired image-to-image translation (I2I) methods achieve strong performance in the visible spectrum, but deteriorate in non-visible modalities such as mid-wave infrared (MWIR) due to severe spectral and radiometric mismatches. In MWIR, object identity is encoded in spatially localized thermal gradients rather than appearance, making conventional perceptual or distribution-matching objectives insufficient for downstream recognition. We address this limitation by proposing a task-aware unpaired translation framework that combines pseudo-pair generation with semantic consistency constraints. Pseudo-paired data are first synthesized using Contrastive Unpaired Translation (CUT) by mapping target-domain images into the source domain. Building on these pairs, we introduce P2PGM, a generative model trained with adversarial and Sobel gradient losses to preserve infrared-specific structural cues, together with a gradient-weighted class activation map (GradCAM)-based semantic loss that aligns task-relevant activations between pseudo-source and generated target images. GradCAM supervision is obtained from a ResNet18 classifier pretrained on real MWIR data, providing domain-specific semantic guidance during translation. On MWIR benchmarks, the proposed framework improves perceptual quality, reducing the Fréchet Inception Distance from 90.10 to 85.21 and increasing SSIM from 0.776 to 0.808. On Cityscapes (Label $$\rightarrow $$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mo>→</mml:mo> </mml:math> Photo), P2PGM achieves superior semantic consistency, improving mIoU from 24.70 to 26.42 and per-class accuracy from 30.70 to 35.47, while reducing FID from 56.40 to 48.52, as measured using a pretrained DRN segmentation network without fine-tuning. Despite emphasizing infrared characteristics, the method generalizes effectively to RGB domains, demonstrating robust task-aware unpaired translation. The framework trades pixel-level supervision for scalable classifier-driven semantic guidance, enabling practical sim-to-real adaptation when dense labels and paired MWIR data are unavailable.