94. A Deep Learning Framework to Measure Eyelid Pigmentation Using Two-dimensional Eye Images of Hereford Crossbred Cattle: From Pixels to Patterns.

Abstract Cancer eye and other ocular problems cause significant economic losses to the cattle industry and are preventable through increased eyelid pigmentation (EP). Previously, EP has been measured by visual assessments, which are subjective, labor-intensive, error-prone, and time-consuming. This study aims to measure EP on a continuous scale from two-dimensional images using machine learning and deep learning approaches. Photographs of left and right eyes were available on 4,257 Hereford and Hereford-cross cattle as part of a multi-state collaboration, where a random set of images (n = 1,978) was selected to develop, train, and validate a deep learning framework. Using 70% training, 20% validation, and 10% test split on images, a Mask R-CNN (Mask Region-Based Convolutional Neural Network) model built in Python with TensorFlow was trained to predict eyeball masks from eye images. The Mask R-CNN model achieved a mean average precision of 1.0 at an IoU threshold of 0.50 on the test set. Once predicted, the eyeball mask was expanded by 5% on all sides using dilation, a morphological operation in the OpenCV library, thereby masking the eyelids along with the eyeball. An arithmetic operation was applied to subtract the eyeball mask from the expanded mask (eyeball + eyelid), yielding the area of the photograph that contained the eyelid. The validated eyelid segmentation step was run on a new, random subset of images (n = 1,398; not used previously). This new subset was utilized to create and train an image quality control (QC) model with the same split ratio as the segmentation step. The QC step used MobileNetV2 architecture and classified the segmented eyelid as “accepted” if complete and “not accepted” if the eyelid was incomplete. An overall accuracy of 0.9493 was achieved on the test dataset, with F1 Scores of 0.9542 (accepted) and 0.9431 (not-accepted). Accepted eyelids (n = 73 predicted out of 78 actual) were fed to the HSV thresholding function using pigmentation thresholds sampled from available images to quantify EP proportion relative to the whole eyelid area, providing an image-derived phenotype (IDP). The accepted images and IDP values were manually validated for image distortion arising from eyelashes, glare from the eyeball affecting eyelid color, and/or shadows due to camera angle or lighting. Manual validation excluded 8 additional images, reducing the QC model's overall accuracy to 0.8913. The current framework identifies segmentation and classification as steps that can be automated but still require further development to address color quantification issues and ensure robust HSV thresholding. Once developed, IDPs will be used in genome-wide association analyses to identify genomic regions that influence EP.