Wireless Capsule Endoscopy Diagnosis Using Prototype Self-Attention and Dynamic Curriculum Learning

Wireless capsule endoscopy is a non-invasive and painless approach for diagnosing gastrointestinal diseases. An automated medical decision support system can significantly enhance clinician efficiency and reduce the incidence of mis-diagnosis when analyzing lesions within the 50,000 to 100,000 image frames generated for each individual. However, only a small number of previous studies have focused on fine-grained recognition characteristics, and very few have simultaneously addressed both fine-grained recognition and class imbalance issues in wireless capsule endoscopy. To address these issues, we propose a novel medical decision support network that includes a prototype attention enhancement module and a dynamic curriculum learning approach. The prototype attention enhancement module improves lesion-sensitive compact representation learning by leveraging cosine similarity between the representation and a prototype memory with multi-class centers. The dynamic curriculum learning adopts triplet loss and weighted cross-entropy, facilitated by a progressive factor that controls between fine-grained representation learning and class-sensitive learning. Through extensive comparative experiments on three public datasets, the proposed network demonstrated competitive performance, outperforming previous methods with an F1-score of 96.6% on the 10-class Kvasir-Capsule dataset, an accuracy of 98.3% on the CAD-CAP dataset, and an accuracy of 93.7% on the mixed KID dataset. Two additional gastrointestinal histopathology datasets confirm the generalization of the proposed network. The code will be available at https://github.com/Xingcun-Li/MDSN-WCE.