A Classifier Model Using Fine-Tuned Convolutional Neural Network and Transfer Learning Approaches for Prostate Cancer Detection

Background: Accurate and reliable classification models play a major role in clinical decision-making processes for prostate cancer (PCa) diagnosis. However, existing methods often demonstrate limited performance, particularly when applied to small datasets and binary classification problems. Objectives: This study aims to design a fine-tuned deep learning (DL) model capable of classifying PCa MRI images with high accuracy and to evaluate its performance by comparing it with various DL architectures. Methods: In this study, a basic convolutional neural network (CNN) model was developed and subsequently optimized using techniques such as L2 regularization, Tanh activation, dropout, and early stopping to enhance its performance. Additionally, a pyramid-type CNN architecture was designed to simultaneously evaluate both fine details and broader structures by combining low- and high-resolution information through feature maps extracted from different CNN layers. This approach enabled the model to learn complex features more effectively. For performance comparison, the developed fine-tuned enhanced pyramid network (FT-EPN) model was benchmarked against models such as Vgg16, Vgg19, Resnet50, InceptionV3, Densenet121, and Xception, which were trained using transfer learning (TL) techniques. It was also compared to next-generation models such as vision transformer (ViT) and MaxViT-v2. Results: The developed fine-tuned model achieved an accuracy rate of 96.77%, outperforming pre-trained TL models and next-generation models like ViT and MaxViT-v2. Among the TL models, Vgg19 achieved the highest accuracy rate at 92.74%. In comparison, ViT achieved an accuracy of 93.55%, while MaxViT-v2 achieved an accuracy of 95.16%. Conclusions: This study presents an optimized FT-EPN model to enhance the performance of DL models for PCa classification, offering a reference solution for future research. This model provides significant advantages in terms of classification accuracy and simplicity and has been evaluated as an effective solution in clinical applications.