A robust deep learning pipeline for multi-class cervical cancer cell identification

Cervical cancer remains a significant global health concern, necessitating accurate and early diagnostic tools. This paper presents a robust deep learning pipeline for the multi-class classification of cervical cancer cells, introducing an optimized YOLO-based architecture enhanced with a novel Attention-Guided Multi-Scale Feature Fusion (AGMS-FF) module. We conduct a comprehensive evaluation, comparing our Full AGMS-FF model against its Baseline YOLOv11 counterpart and established convolutional neural networks including EfficientNet-B0, MobileNetV3, and ResNet18. Experiments were rigorously conducted on two distinct cervical cell datasets to assess model robustness and generalization. On Dataset 1 (9,500 images), the Full AGMS-FF model achieved the highest accuracy of 0.9256 and an exceptional Macro AUC of 0.9910, outperforming EfficientNet-B0 (0.8330 accuracy), MobileNetV3 (0.8028), and ResNet18 (0.7324). The Baseline YOLOv11 also demonstrated strong performance (0.9235 accuracy, 0.9910 Macro AUC). On the more challenging Dataset 2 (4,966 images), the Full AGMS-FF model again led with an accuracy of 0.8471 and a Macro AUC of 0.9791, with Baseline YOLOv11 at 0.8310 accuracy and 0.9774 Macro AUC. Both YOLO-based models consistently demonstrated remarkable performance across a wide spectrum of cervical cell classes, showing particularly high F1-scores for both benign and certain abnormal cell types. In contrast, other CNNs struggled notably with less frequent or more challenging pathological variations. Our findings highlight the profound capability of the enhanced YOLO architecture in achieving high-precision, multi-class classification across different data distributions, offering a promising avenue for clinical diagnostic support.