S1298 Barlow Twins Self-Supervised Vision Transformer for Robust Classification of Gastrointestinal Endoscopic Images

Introduction: Accurate endoscopic pathology classification depends on time-consuming expert annotations that limit large-scale deployment. We pretrained a Vision Transformer with a decorrelation-based self-supervised framework on ∼100 000 unlabeled images to learn high-fidelity representations without manual labels. Methods: We leveraged the HyperKvasir repository to assemble 99 417 unlabeled endoscopic images. A self-supervised encoder based on the Barlow Twins architecture processed paired augmentations of each image to learn invariant, non-redundant feature representations. We extracted high-level embeddings with a Vision Transformer backbone and refined them through a projection network optimized by a decorrelation loss. Following pretraining, we fine-tuned the encoder with a multilayer perceptron classifier on 10 662 manually labeled images spanning 23 gastrointestinal conditions. We set aside 20% of the labeled data for testing. Performance metrics included accuracy, precision, recall, weighted F1-score, and area under the receiver operating characteristic curve. Results: The encoder achieved 87.6% accuracy and a weighted F1-score of 0.87 on the held-out test set. It maintained a mean area under the curve (AUC) of 0.99 across all 23 categories. Esophagitis, Barrett’s esophagus, and colonic polyps attained class-level F1-scores above 0.95. Performance remained consistent across both prevalent and rare conditions. t-SNE projections revealed distinct, well-separated clusters, and the confusion matrix exhibited minimal misclassification among closely related lesions. Such robust results demonstrate that decorrelation-based self-supervision can deliver clinically meaningful representations while dramatically reducing the need for extensive manual annotation. Conclusion: Fine-tuning on 10 662 expert-annotated images yielded 87.6% accuracy, a weighted F1-score of 0.87, and a mean AUC of 0.99 across 23 gastrointestinal conditions, with Barrett’s esophagus and colonic polyps exceeding class-specific F1-scores of 0.95. Feature-space visualization and confusion-matrix analysis confirmed clear separation of disease categories and minimal overlap among similar lesions. By slashing annotation demands, decorrelation-based self-supervision lays a scalable, practical foundation for deploying artificial intelligence-driven diagnostic support in everyday endoscopic practice.