Color Representation and Prediction Uncertainty in Unsupervised Retinal Vessel Segmentation

This study investigates the influence of color representation, preprocessing, and thresholding strategies on unsupervised retinal vessel segmentation using computationally efficient classical image processing techniques. Nineteen image channels derived from RGB, HSV, HSL, YUV, YCrCb, CIELAB, and grayscale representations were evaluated in combination with five thresholding algorithms on the DRIVE dataset. The results demonstrate that luminance-dominant channels consistently outperform chrominance-based representations. The Moments (Tsai) thresholding algorithm provided the most robust performance across image representations, while contrast enhancement using CLAHE and white top-hat filtering significantly improved segmentation quality. In addition, the weighted combination of RGB channels was characterized by a dominant green-channel contribution (wG ≈ 0.88–0.90) and a minimal blue-channel contribution (wB ≈ 0–0.02), reflecting the spectral properties of retinal vasculature. The study highlights the central role of luminance information and demonstrates that task-specific optimization of color representations improves both segmentation accuracy and reliability. Although deep learning methods currently achieve state-of-the-art performance in retinal vessel segmentation, the results show that substantial optimization remains possible using classical, interpretable, and computationally efficient approaches.