Comparison of the Aireen System with Telemedicine Evaluation by an Ophthalmologist – A Real-World Study

Martin Šín,1 Renata Ženíšková,1 Martin Slíva,2 Kamila Dvořák,2,3 Jozefína Vaľková,2 Jan Bayer,2 Barbora Karasová,2 Jan Tesař,1 Dana Fillová,4 Martin Prázný5,6 1Department of Ophthalmology, Military University Hospital Prague, 1st Faculty of Medicine, Charles University, Prague, Czech Republic; 2Aireen a.s., Prague, Czech Republic; 3Department of Natural Sciences, Faculty of Biomedical Engineering, Czech Technical University, Prague, Czech Republic; 4Eye Centre Prague a.s., Prague, Czech Republic; 5 3rd Department of Internal Medicine, General University Hospital in Prague, Prague, Czech Republic; 6 3rd Department of Medicine - Department of Endocrinology and Metabolism, 1st Faculty of Medicine, Charles University, Prague, Czech RepublicCorrespondence: Martin Šín, Department of Ophthalmology, Military University Hospital Prague, 1st Faculty of Medicine, Charles University, U Vojenské nemocnice 1200, 169 02, Praha 6, Prague, Czech Republic, Email [email protected]: This study aimed to compare general ophthalmologists, retina specialists, and Aireen AI screening system with the clinical reference standard of a three-member high-level expert committee for diabetic retinopathy (DR) in the evaluation of fundus images for DR.Patients and Methods: The study was designed as a diagnostic, multicenter, cross-sectional, non-randomized diagnostic study. The cohort included in the clinical investigation consisted of 1274 patients with diabetes mellitus (DM) type I or II. Each patient underwent one-field fundus photography using a non-mydriatic camera to assess findings of DR. One hundred and nineteen subjects (9.3%) were excluded from the clinical investigation based on Aireen system assessment. In the clinical investigation, all images were assessed at three independent levels of evaluation: 1) general ophthalmologists (GO) – without subspecialty training in the retina; 2) retina specialists (RS); and 3) system Aireen. In cases where there may be disagreements amongst groups, the image is referred for assessment by the Diabetic Retinopathy Board (DRB).Results: The overall prevalence of any DR was 31.9% (368 cases out of 1154 DM), according to the DRB. Overall concordance between AI system Aireen and GO and RS assessments in the detection of DR from fundus photography occurred in 734 cases (63.6%). The number of disagreements between Aireen system, GO and RS evaluation occurred in 420 (36.4%) cases. Sensitivity for GO was 87.0% (95% CI: 83.6; 90.4), for RS was 82.9% (95% CI: 79.1; 86.7), and for AI system Aireen was 92.1% (95% CI: 89.3; 94.9). Specificity was 76.5% (95% CI: 73.5; 79.5), 81.2% (95% CI: 78.5; 83.9), and 90.7% (95% CI: 88.7; 92.7) for GO, RS and AI system Aireen, respectively.Conclusion: This real-world study illustrates the potential use of AI system Aireen in screening for DR. It exhibits higher sensitivity and specificity compared to telemedicine evaluation of one field fundus image.Keywords: diabetic retinopathy, artificial intelligence, screening, fundus image