Multi-modal retinal disease diagnosis based on fundus photography and OCT images

Retinal diseases are a serious threat to human visual health and their early diagnosis is crucial. Currently, most of the retinal disease diagnostic algorithms are based on a single imaging modality of fundus color photography (FCP) or optical coherence tomography (OCT). These methods can only reflect retinal diseases to a certain extent, ignoring the specificity of modalities between different imaging modalities. In this research, a new multi-scale feature fusion network (MSFF-Net) model for multi-modal retinal image diagnosis is proposed. The MSFF-Net model employs a dual-branch architecture design, enabling efficient learning and extraction of multi-modal feature information related to retinal diseases from CFP and OCT images. MSFF-Net improves disease diagnosis by combining multi-scale features of CFP and OCT images. When evaluated on challenging datasets, the model achieved an accuracy of 95.00% and an F1-score of 95.24% for retinal disease diagnosis. Even under low-quality dataset conditions, it maintained robust performance, with diagnostic accuracy and F1-scores of 71.50% and 71.73%, respectively. In addition, the MSFF-Net model outperformed eight state-of-the-art single and multi-modal models in the comparison experiments. The proposed MSFF-Net model provides ophthalmologists with a more accurate and efficient diagnostic pathway that helps them detect and treat retinal diseases earlier.