A Multimodal Information-based Knowledge Distillation Method for Small Facial Acne Detection

This study addresses the challenges of feature representation and computational complexity in small facial acne detection by proposing a multi-modal knowledge distillation method. An enhanced YOLOv8s-based teacher model is proposed, integrating a transformer architecture into the backbone to improve global feature capture and a novel multi-scale attention mechanism in the neck network to enhance small acne feature representation. The student model is a lightweight version of the teacher model, obtained by replacing standard convolution with depthwise separable convolution. Additionally, a multi-modal distillation approach leveraging spatial, channel, and correlation information is proposed to overcome limitations of single-information distillation, enabling efficient training of student models. Experiments show that the proposed method achieves mAP of 31.52% on ACNE4K and 24.1% on ACNE04, with 7.83M parameters and 61.06G FLOPs, reducing parameters by 45% and computational costs by 50% compared to the teacher model while maintaining comparable accuracy.