Comparative Analysis of Machine Learning Methods on Cervical Cytology Images

Cervical cancer is the fourth most common cancer in women worldwide, although it is largely preventable through early diagnosis. Expert reading of Pap smear cytology is time-consuming and subject to inter-observer variability. This study compares three approaches on 7,341 Pap smear images from four merged public datasets labelled with five Bethesda classes (NILM, ASCUS, LSIL, HSIL, SCC): classical machine learning with handcrafted colour, texture and shape features; classical machine learning with deep features extracted from ImageNet-pretrained convolutional neural networks (EfficientNet-B0, ResNet50); and an end-to-end fine-tuned EfficientNet-B0. An MD5 hash check showed that 933 images belonged to duplicate groups and that 112 hashes appeared with conflicting labels; 6,620 images remained after cleaning. Twenty-six configurations built with eight classifiers were evaluated on a shared stratified test set of 993 images. The fine-tuned EfficientNet-B0 performed best, with 84.99% accuracy, a macro F1 of 0.843, a ROC-AUC of 0.968 and a quadratic weighted kappa of 0.914. Among the classical pipelines, the SVM on EfficientNet-B0 features gave the best result (macro F1 0.746). In a normal-versus-abnormal screening setting, the best model reached 95.2% sensitivity and 85.1% specificity. ASCUS was the hardest class to separate. Evidence that source-specific image properties may lead to shortcut learning is also discussed. Preprint, not peer reviewed. Code: https://www.kaggle.com/code/tolga0z/cervical-cancer-ml-model-comparison