Transformer-Based Deep Learning Framework for Automated Lesion Detection in Capsule Endoscopy

GOALS: To compare a vision transformer with 2 convolutional neural network architectures for multiclass lesion classification in capsule endoscopy images. BACKGROUND: Manual review of capsule endoscopy is time-consuming and subject to interobserver variability. Deep learning can automate lesion recognition; however, most prior capsule endoscopy work evaluates a small number of classes, and systematic comparisons between transformer and convolutional architectures across many lesion categories are limited. STUDY: Two publicly available data sets (SEE-AI and Kvasir-Capsule) were merged and preprocessed to create a 21-class image data set (∼58,000 frames). Images were resized to 224 × 224 pixels and split using stratified sampling into training (n=40,587), validation (n=8696), and test (n=8696) sets. A pretrained Vision Transformer, DenseNet121, and ResNet50 were fine-tuned using categorical cross-entropy loss and Adam optimization, with early stopping. Performance was assessed using accuracy, macroaveraged precision, recall, F1 Score, and the area under the receiver operating characteristic curve. RESULTS: On the independent test set, the vision transformer achieved 92.2% accuracy with macroaveraged precision/recall/F1-score of 0.92 and an area under the receiver operating characteristic curve of 0.99. DenseNet121 achieved 74.0% accuracy (F1-score 0.78; area under the receiver operating characteristic curve 0.85). ResNet50 achieved 38.0% accuracy (F1-score 0.40; area under the receiver operating characteristic curve 0.55). CONCLUSIONS: In this merged 21-class capsule endoscopy image data set, the vision transformer achieved higher frame-level classification performance than DenseNet121 and ResNet50 under the present experimental conditions. Importantly, the data set was split at the image level rather than at the patient or procedure level, because frames from the same examination may be correlated; therefore, performance figures likely reflect benchmark results on this frame-level public data set and should not be interpreted as estimates of patient-level generalization or as evidence of definitive architectural superiority. These findings support further evaluation of transformer-based approaches, but grouped reanalysis, external validation, and workflow-oriented studies are required before clinical implementation.