AI-Based Multimodal Diabetic Retinopathy Prognostic System with Progression Twin Generation

This paper presents an AI-Based Multimodal Diabetic Retinopathy Prognostic System with Progression Twin Generation for intelligent diabetic retinopathy diagnosis, prognosis analysis, and future retinal condition visualization. The proposed system integrates retinal fundus image analysis, clinical-data-based prediction, multimodal fusion, prognostic AI, Grad-CAM explainability, and retinal digital twin generation into a unified healthcare decision-support framework. The retinal image analysis module employs a CNN-based deep learning architecture for automated diabetic retinopathy severity classification across five classes including No DR, Mild, Moderate, Severe, and Proliferative DR. To improve diagnostic reliability, structured patient clinical parameters such as HbA1c, blood pressure, BMI, fasting glucose level, diabetes duration, cholesterol, and triglycerides are analyzed using a Random Forest-based machine learning model. Both image-based and clinical-data-based predictions are integrated using a multimodal fusion mechanism for comprehensive diabetic retinopathy severity estimation. The proposed framework additionally incorporates Grad-CAM (Gradient-weighted Class Activation Mapping) visualization to improve model interpretability and explainability. Grad-CAM heatmaps highlight important retinal regions and lesion areas contributing to the prediction process, enabling improved transparency and assisting users in understanding CNN decision behavior. A lightweight retinal digital twin generation module is further utilized to simulate possible future retinal progression patterns based on the patient’s current retinal condition. The system also generates prognostic recommendations including treatment guidance, follow-up scheduling, lifestyle suggestions, and disease progression interpretation. All components are integrated into a Flask-based web application for real-time healthcare deployment and user interaction. Experimental evaluation demonstrates the practical feasibility of combining deep learning, multimodal AI, explainable AI, prognostic intelligence, and digital twin visualization into a unified intelligent ophthalmic healthcare platform.