SAGE: An Expert-Annotated South Asian GI Endoscopy Dataset for Multimodal Learning and Hallucination Analysis

Gastrointestinal cancers represent a growing health burden in the South Asian region, driven largely by rapid changes in socio-economic conditions and lifestyle habits. However, early diagnosis remains limited by inadequate equipment, financial resources, and scarce GI expertise. AI-assisted diagnosis and report generation show great promise in alleviating this problem by providing non-specialist healthcare workers the technical expertise to perform diagnosis. Yet, almost all open-source, publicly available datasets are predominantly collected from the European region, with minimal representation from the South Asian region. The lack of open-source GI datasets from diverse geographic regions has made it difficult to assess whether population bias is present in existing models, and to develop geographically inclusive AI tools for automated GI diagnosis. To address this gap, we introduce SAGE: An Expert-Annotated South Asian GI Endoscopy dataset for Multimodal Learning and Hallucination Analysis, for image captioning, multi-label classification, and visual question answering (VQA) tasks. It consists of 1,300 images, their captions along with hallucination tag, 18 labels and 14,276 question-answer pairs making it well-suited for diverse range of tasks including classification, benchmarking, and fine-tuning large multimodal models (LMMs). We further conducted benchmarking of task-specific models, like multi-class classifiers on the effect of population shift which reveals that such models suffers the most with an average F1 score drop of 54 points on South Asian dataset. Further, benchmarking of contemporary LMMs reveals a substantial drop in the average GREEN score for anatomical landmark detection (0.308) and abnormality detection (0.410). We open-source our dataset under the CC BY-SA 4.0 license, and we hope to encourage others to contribute toward more inclusive dataset representation and help counteract population bias in medical AI. The code is publicly available at https://github.com/bhattarailab/SAGE, and the dataset at https://www.synapse.org/SAGE.