Artificial intelligence based prediction of first recurrence in neovascular age related macular degeneration with validation by 19 experts

This study aimed to investigate the value and difference in predictive performance between ophthalmologists and a previously developed and validated artificial intelligence (AI) model, and to evaluate how AI assistance influences expert decision-making in reliably assessing recurrence prediction of neovascular age-related macular degeneration (nAMD) after anti-vascular endothelial growth factor (VEGF) treatment. 19 experts (nine retinal specialist ophthalmologists and ten non-retinal specialist ophthalmologists) predicted the first recurrence of nAMD within three months based on optical coherence tomography (OCT) images and clinical information. Predictions were made in five sessions with increasing information availability. The AI model used in this study had been developed and validated in our earlier work, and it predicted recurrence using baseline and after the loading phase OCT images. We compared the area under the receiver operating characteristic curve (AUROC), Fleiss' kappa, and Delong's test between expert groups and the AI algorithm. The study included 149 eyes of 130 patients. The AI model achieved an AUROC of 0.744 (95% confidence interval, 0.665-0.822). Expert performance improved across sessions, with AUROCs ranging from 0.562 ± 0.034 to 0.679 ± 0.049. No significant differences were observed between expert groups based on experience or subspecialty. AI-supported decisions showed slightly improved performance in predicting nAMD recurrence compared to human experts, regardless of clinical experience. These results suggest the potential of AI-assistance in predicting recurrence and optimizing treatment strategies for nAMD, which could significantly improve patient counseling and management. This study also highlights the novel contribution of evaluating the impact of AI assistance on ophthalmologists' decision-making in nAMD recurrence prediction.