Automated Detection and Classification of Strabismus in Smartphone Images

Abstract Purpose Accurate detection of strabismus from smartphone images remains challenging, particularly in distinguishing true strabismus from pseudo-strabismus, in which apparent ocular misalignment is caused by facial anatomy rather than abnormal eye position. This study developed and evaluated a fully automated smartphone-based approach that explicitly models interocular asymmetry between the corneal light reflex (CLR) and pupil center for strabismus detection and classification. Methods A Mask R-CNN framework was trained to localize pupil centers and CLR positions from facial images. Geometric asymmetry parameters derived from the CLR-pupil relationship in the two eyes were used to detect strabismus and classify detected cases as horizontal or vertical deviations, with horizontal cases further classified as esotropia or exotropia. The final evaluation used a separate set of 205 clinically labeled smartphone images comprising orthotropic children, pseudo-strabismus cases, and children with strabismus. Results Of 110 orthotropic images, 107 were correctly classified as orthotropic, while 87 of 95 strabismic images were correctly identified. Overall strabismus detection accuracy was 94.63%, with 96.67% precision, 91.58% sensitivity, 97.27% specificity, and an F1-score of 94.06%. Horizontal-versus-vertical classification achieved 95.40% accuracy, and classification of horizontal strabismus as esotropia or exotropia achieved 96.15% accuracy. None of the 12 pseudo-strabismus cases in the test set was misclassified as strabismus. Conclusion Explicit modeling of interocular CLR-pupil asymmetry enables accurate automated detection and classification of strabismus from smartphone images and may help distinguish true strabismus from pseudo-strabismus. Following prospective clinical validation, this approach has potential to support accessible smartphone-based strabismus screening.