Self-Supervised Learning for Multi-Category Endoscopy Classification and Data Quality Evaluation Using Masked Autoencoders

In upper and lower endoscopy (EN), gastroenterologists must identify multiple anatomical landmarks, lesions, and indicators of patient preparation, requiring the recognition of patterns that may elude the human eye, leading to 6-28% of lesions being missed. Computer-assisted second reader strategies may aid specialist by accurately differentiating multiple patterns. However, labeled collections are scarce for training, though some provide unlabeled frames. This paper leverages a large-scale, unlabeled dataset to train a self-supervised learning (SSL) approach, a masked autoencoder, to identify multiple EN categories. The pretrained masked-encoder is then supervised fine-tuning on a smaller, labeled subset, in a multiclass classification task. The performance of this method was evaluated on two EN datasets: HyperKvasir, which contains 99,417 unlabeled and 10,662 labeled frames, and GastroVision, which includes 8,000 labeled frames, achieving F-scores of 89.23% in 16 EN categories and 77.40% in 22, respectively. The proposed approach demonstrates the effectiveness of a SSL approach in EN, and improves classification performance with data quality control by leveraging learned embeddings to identify and remove duplicate images within the dataset. We also provide code and materials related to this work, which can be found at https://github.com/DiegoBravoH/SSL-Endoscopy-MAE.git.