CEMIL: Contextual Attention Based Efficient Weakly Supervised Approach for Histopathology Image Classification

Multiple Instance Learning (MIL) has shown potential for analyzing Whole Slide Images (WSIs) in digital pathology, but it faces challenges related to redundant information learning and generalization due to limited supervision and the computational complexity of Gigapixel WSIs. Many MIL-based methods apply a small weight matrix to all WSI patches. In this study, we focus on developing computationally efficient models that improve MIL-based WSI classification by processing fewer patches while improving performance. We propose an attention-based approach using knowledge distillation, where a compute-intensive “instructor” model analyzes all WSI patches to train a resourceefficient “learner” model, which considers only a subset of patches. Comprehensive evaluations on four cancer subtype datasets—TCGA-BRCA, TCGA-NSCLC, TCGA-RCC, and PANDA—demonstrate that an “observe-everything” instructor can effectively train an “observe-minimally” learner network. Overall, our proposed learner network enhances performance by 4% compared to the state-of-the-art, while reducing inference time by 45% and FLOPs by approximately 88%.