Statistical Evaluation of Smartphone-Based Automated Grading System for Ocular Redness Associated with Dry Eye Disease and Implications for Clinical Trials

John D Rodriguez,1 Adam Hamm,2 Ethan Bensinger,1 Samanatha J Kerti,2 Paul J Gomes,3 George W Ousler III,3 Palak Gupta,1 Carlos Gustavo De Moraes,3,4 Mark B Abelson1,3,5 1Andover Eye Institute, Andover, MA, 01810, USA; 2SDC, Tempe, AZ, 85288, USA; 3Ora, LLC, Andover, MA, 01810, USA; 4Columbia Medical Center, New York, NY, 10032, USA; 5Harvard Medical School, Boston, MA, 02115, USACorrespondence: Paul J Gomes, Allergy & Blepharitis, Ora, LLC, 138, Haverhill Street, Suite 102, Andover, MA, 01810, USA, Tel +1-978-685-8900, Fax +1-978-689-0020, Email [email protected]: This study introduces a fully automated approach using deep learning-based segmentation to select the conjunctiva as the region of interest (ROI) for large-scale, multi-site clinical trials. By integrating a precise, objective grading system, we aim to minimize inter- and intra-grader variability due to perceptual biases. We evaluate the impact of adding a “horizontality” parameter to the grading system and assess this method’s potential to enhance grading precision, reduce sample size, and improve clinical trial efficiency.Methods: We analyzed 29,640 images from 450 subjects in a multi-visit, multi-site clinical trial to assess the performance of an automated grading model compared to expert graders. Images were graded on a 0– 4 scale, in 0.5 increments. The model utilizes the DeepLabV3 architecture for image segmentation, extracting two key features—horizontality and redness. The algorithm then uses these features to predict eye redness, validated by comparison with expert grader scores.Results: The bivariate model using both redness and horizontality performed best, with a Mean Absolute Error (MAE) of 0.450 points (SD=0.334) on the redness scale relative to expert scores. Expert graded scores were within one unit of the mean grade in over 85% cases, ensuring consistency and optimal training set for the predictive model. Models incorporating both features outperformed those using only redness, reducing MAE by 5– 6%. The optimal generalized model improved predictive accuracy with horizontality such that 93.0% of images were predicted with an absolute error less than one unit difference in grading.Conclusion: This study demonstrates that fully automating image analysis allows thousands of images to be graded efficiently. The addition of the horizontality parameter enhances model performance, reduces error, and supports its relevance to specific Dry Eye manifestations. This automated method provides a continuous scale and greater sensitivity to treatment effects than standard clinical scales.Keywords: dry eye hyperemia, deep-learning conjunctiva segmentation, redness, automated grading