Enhancing Glaucoma Diagnosis through Vision-Language Models and Large Language Model Descriptions

Deep learning approaches have been widely studied for their effectiveness in glaucoma detection using fundus images. For deep learning models to perform well, high-quality labeled data is essential for training the models. However, as annotating medical images requires expertise and is a costly process, the size of the datasets are often quite small. In this paper, we explore the advantage of leveraging a large-scale multi-modal foundation model along with textual descriptions generated by a large language model (LLM) in order to classify fundus images. Our findings indicate that incorporating textual information into the model enhances its performance by increasing diversity. However, excessive textual information can lead to poor associations, negatively impacting the model's performance.