Analyzing the Impact of Photometric Data Augmentation on Medical Instance Segmentation Performance

This study analyzes the effect of photometric data augmentation strategies on medical instance segmentation using the Kvasir-SEG dataset. Mask R-CNN X101-FPN and YOLOv8l-seg are adopted as representative two-stage and one-stage segmentation models. Although data augmentation is commonly employed to enhance robustness in image classification, its systematic evaluation in instance segmentation remains limited due to the need to preserve pixel-level mask integrity during transformations. In this work, eight photometric augmentation techniques, including Hue, Saturation, Grayscale, Brightness, Contrast, Noise, Blur and Cutout, are applied both individually and through structured multi-level combinations. Each augmentation is evaluated within single, double, triple, and full pipelines. Segmentation performance is measured using mean Average Precision (mAP) based on the COCO evaluation protocol. The experimental results show that color-based augmentations provide more reliable accuracy improvements than distortion-based methods in polyp segmentation tasks, while excessive augmentation depth may slow convergence and restrict performance gains. This study presents a systematic analysis of augmentation depth and diversity and offers practical guidance for designing effective augmentation pipelines in medical instance segmentation.