Detection of benign prostatic hyperplasia using RGB prostate images and deep learning

This study investigates the identification of Benign Prostatic Hyperplasia (BPH) through a deep learning-based analysis of RGB prostate histopathological images. Adaptive Contrast Limited Adaptive Histogram Equalization (CLAHE) is selectively applied to the L-channel in the LAB color space to enhance tissue visibility while preserving chromatic fidelity. At the architectural level, Convolutional Neural Networks (CNNs) are integrated with Bidirectional Long Short-Term Memory (BiLSTM) layers, enhanced further through spatial and temporal attention mechanisms. This hybrid design facilitates both localized pattern recognition and the modeling of long-range contextual dependencies across tissue regions. To mitigate class imbalance and prevent overfitting, the training regime incorporates two key strategies: an adaptive focal loss function and a comprehensive image augmentation protocol. The proposed model achieved an AUC of 0.7220 on the validation set and an AUC of 0.73 on the test set. While the precision for normal tissue classification remained high, the recall for BPH detection highlighted the need for improvement in sensitivity. The proposed CNN-BiLSTM-Attention architecture demonstrates potential as a diagnostic aid in digital pathology, offering interpretable insights and forming a foundation for enhancing histological classification systems. Future work will focus on improving recall performance for BPH detection and expanding the architecture to support multi-class prostate disease grading frameworks. This study utilizes an RGB histopathological dataset consisting of 176 prostate images, each appropriately annotated. The model demonstrates moderate classification performance and a moderate true-positive rate for detecting Normal samples. The model, however, has a low sensitivity in the detection of the cases of BPH as indicated by the relatively low recall values.