TV-LSTM: Multimodal Deep Learning for Predicting the Progression of Late Age-Related Macular Degeneration Using Longitudinal Fundus Images and Genetic Data

Age-related macular degeneration (AMD) is the leading cause of blindness in developed countries. Predicting its progression is crucial for preventing late-stage AMD, as it is an irreversible retinal disease. Both genetic factors and retinal images are instrumental in diagnosing and predicting AMD progression. Previous studies have explored automated diagnosis using single fundus images and genetic variants, but they often fail to utilize the valuable longitudinal data from multiple visits. Longitudinal retinal images offer a dynamic view of disease progression, yet standard Long Short-Term Memory (LSTM) models assume consistent time intervals between training and testing, limiting their effectiveness in real-world settings. To address this limitation, we propose time-varied Long Short-Term Memory (TV-LSTM), which accommodates irregular time intervals in longitudinal data. Our innovative approach enables the integration of both longitudinal fundus images and AMD-associated genetic variants for more precise progression prediction. Our TV-LSTM model achieved an AUC-ROC of 0.9479 and an AUC-PR of 0.8591 for predicting late AMD within two years, using data from four visits with varying time intervals.