Deep Learning for Comprehensive Analysis of Retinal Fundus Images: Detection of Systemic and Ocular Conditions

The retina offers a unique window into both ocular and systemic health, motivating the development of AI-based tools for disease screening and risk assessment. In this study, we present a comprehensive evaluation of six state-of-the-art deep neural networks, including convolutional neural networks and vision transformer architectures, on the Brazilian Multilabel Ophthalmological Dataset (BRSET), comprising 16,266 fundus images annotated for multiple clinical and demographic labels. We explored seven classification tasks: <i>Diabetes</i>, <i>Diabetic Retinopathy (2-class)</i>, <i>Diabetic Retinopathy (3-class)</i>, <i>Hypertension</i>, <i>Hypertensive Retinopathy</i>, <i>Drusen</i>, and <i>Sex</i> classification. Models were evaluated using precision, recall, F1-score, accuracy, and AUC. Among all models, the Swin-L generally delivered the best performance across scenarios for <i>Diabetes</i> (AUC = 0.88, weighted F1-score = 0.86), <i>Diabetic Retinopathy (2-class)</i> (AUC = 0.98, weighted F1-score = 0.95), <i>Diabetic Retinopathy (3-class)</i> (macro AUC = 0.98, weighted F1-score = 0.95), <i>Hypertension</i> (AUC = 0.85, weighted F1-score = 0.79), <i>Hypertensive Retinopathy</i> (AUC = 0.81, weighted F1-score = 0.97), <i>Drusen</i> detection (AUC = 0.93, weighted F1-score = 0.90), and <i>Sex</i> classification (AUC = 0.87, weighted F1-score = 0.80). These results reflect excellent to outstanding diagnostic performance. We also employed gradient-based saliency maps to enhance explainability and visualize decision-relevant retinal features. Our findings underscore the potential of deep learning, particularly vision transformer models, to deliver accurate, interpretable, and clinically meaningful screening tools for retinal and systemic disease detection.