Activation Map-based Knowledge Distillation for Real-time Cervical OCT Image Classification

Cervical cancer is a significant global health concern for women. Optical coherence tomography (OCT) offers a non-invasive, high-resolution imaging method for cervical examinations. The clinical need for real-time AI-aided diagnosis in low-resource settings necessitates model compression for deep learning models. Therefore, we develop a novel activation map-based knowledge distillation (AMKD) framework to address this issue. The AMKD framework ensures that the compressed (student) model achieves classification performance approximating that of the teacher model while maintaining the same activation area, enhancing interpretability for gynecologists. Due to the lack of high-quality annotated data, we also utilize unlabeled images for self-supervised distillation pre-training to improve model performance. On an internal dataset, the compressed models based on ResNet, ConvNeXt, and Swin-Transformer outperformed existing knowledge distillation frameworks while demonstrating better interpretability. On two external validation sets, the best compressed (ResNet) model surpassed the average performance of four medical experts in sensitivity and negative predictive value for cervical OCT volume classification, showcasing enhanced lesion detection capabilities. For a cervical OCT image of 760 × 1200 pixels, the compressed ResNet model with 1.37 M parameters uses 1.13 GB GPU memory during inference and predicts the input image in 0.06 seconds, potentially improving real-time detection of cervical lesions in resource-limited clinical settings.