Modality-mapping-guided customized CycleGAN filtering to enhance narrow-band imaging for improved colorectal lesion segmentation

Narrow-band imaging (NBI) improves visualization of mucosal microvasculature and is widely used for colorectal lesion characterization. However, variations in vascular contrast and structural clarity can limit both clinical interpretation and automated analysis. In this work, we propose a customized CycleGAN framework to enhance NBI images using generators trained through unpaired translation between white light (WL) and NBI modalities. During training, the generators learn modality-specific mappings that capture vascular and mucosal characteristics of NBI. We hypothesize that applying this learned transformation to real NBI images can reinforce subtle vascular patterns and improve structural representation. To better preserve fine anatomical details, the CycleGAN architecture was modified by replacing mean absolute error with structural similarity index (SSIM) in cyclic consistency loss and introducing feature concatenation within generator residual blocks. Two generator configurations with 6 and 9 residual blocks were evaluated. The enhanced images showed increased entropy and chromatic contrast relative to the original NBI images. The deeper model demonstrated improved segmentation performance across diverse segmentation architectures, achieving higher Dice coefficients and Jaccard indices with comparable F2 scores. Further lesion class-wise results indicate that it can effectively enhance NBI images and consistently improve lesion segmentation across lesion classes and thereby classification performance.