Cervical cytology screening using the fused deep learning architecture with attention mechanisms

Cervical cancer remains a significant global health concern. Given the disparity between limited medical resources and the requisite professional personnel, the coverage of cervical screening is inadequate, particularly in underdeveloped areas. Computer-assisted liquid-based cytology diagnostic systems offer favorable solutions. Detection of small nuclei within a complex liquid-based environment poses a challenge, exacerbated by the restricted availability of manual annotations. In this study, we propose FuseDLAM, a comprehensive computer-aided diagnostic system, which employs enhanced YOLOv8 with transformers for rapid localization of individual squamous epithelial cells. We leverage artificial intelligence-generated content techniques for data augmentation, effectively reducing the need for costly manual annotations. By integrating multiple deep convolutional neural network models with self-attention mechanisms, the system extracts crucial features from cell nuclei. These features are then fused through a fully connected layer to facilitate robust cell classification. FuseDLAM achieves an F1-score of 99.3 % on the public SIPaKMeD dataset, demonstrating comparability with state-of-the-art approaches. It also proves its practical applicability in real-world clinical scenarios, achieving an F1-score of 91.2 % in identifying abnormal cervical squamous cells. Additionally, ablation experiments in both datasets validate the model's effectiveness. This underscores its potential for widespread application in medical imaging tasks. • This study for the first time applies the AIGC technique in cervical cancer-related research. • The proposed model is specifically tailored for the comprehensive liquid-based cytology diagnostic system, encompassing the segmentation and classification of cervical squamous epithelial nuclei. • The proposed model improves the classification of small targets in images through the joint analysis of multiple deep learning models with tailored enhanced structures. • The proposed model demonstrates practical applicability and effectiveness in publicly available datasets and real-world clinical scenarios. • The present model may inspire other medical imaging tasks, particularly in the areas of segmentation, feature extraction, and dataset augmentation.