DeepResVit: A Hybrid Deep Learning Approach for Ovarian Cancer Classification with XAI

Ovarian cancer is one of the most prevalent cancers in women and primarily results from the uncontrollable growth and division of abnormal cells in the ovary. Early detection is essential for improving survival rates, yet traditional diag-nostic techniques struggle to detect the disease at an early stage. This study aims to address this challenge by proposing a robust hybrid deep learning model named DeepResVit, which integrates both the pre-trained ResNet-152 and Vision Transformer (ViT) architectures for ovarian cancer classification. Several preprocessing techniques, such as data augmentation, image resizing, random horizontal flip, and random vertical flip were applied to improve the image quality. The model is further enhanced with explainable artificial intelligence (XAI) techniques, such as Grad-CAM++, to improve the transparency and interpretability of predictions. In experimental evaluations, DeepResVit achieved outstanding performance with a test accuracy of 98.65%, precision of 98.65%, recall of 98.68%, and F1-score of 98.65%. Custom-connected layers, data augmentation, and L1 regularization strengthened the model's robustness across diverse histopathological images. These results demonstrate the effectiveness of DeepResVit in accurately detecting ovarian cancer at an early stage while providing clear insights into the model's decision-making process.