Adapting SAM for Visible-Light Pupil Segmentation Baseline

Pupil segmentation in visible-light (RGB) images presents unique challenges due to variable lighting conditions, diverse eye colors, and poor contrast between iris and pupil, particularly in individuals with dark irises. While near-infrared (NIR) imaging has been the traditional solution for eye-tracking systems, the accessibility and practicality of RGB-based solutions make them attractive for widespread adoption in consumer devices. This paper presents a baseline for RGB pupil segmentation by adapting the Segment Anything Model (SAM). We introduce a multi-stage fine-tuning approach that leverages SAM’s exceptional generalization capabilities, further enhancing its elemental capacity for accurate pupil segmentation. The staged approach consists of SAM-BaseIris for enhanced iris detection, SAM-RefinedIris for improving iris segmentation with automated bounding box prompts, and SAM-RefinedPupil for precise pupil segmentation. Our method was evaluated on three standard visible-light datasets: UBIRIS.v2, I-Social DB, and MICHE-I. The results demonstrate robust performance across diverse lighting conditions and eye colors. Our method achieves near SOTA results for iris segmentation and attains mean mIOU and DICE scores of 79.37 and 87.79, respectively, for pupil segmentation across the evaluated datasets. This work establishes a strong foundation for RGB-based eye-tracking systems and demonstrates the potential of adapting foundation models for specialized medical imaging tasks.