Attention-Enhanced UNet for Automated Gleason Score 3 Tumor Segmentation in Histopathological Whole Slide Images

The Gleason grading system, applied to histopathology images, serves as the most significant prognostic marker for Prostate cancer (PCa). Pathologists evaluate Gleason grades by analyzing tumor structural growth patterns from whole slide images (WSIs) of tissue specimens. However, this is time consuming process and introduces an inherent subjectivity when evaluated by different pathologists. PCa segmentation in WSIs is challenging due to subtle malignant patterns, gigapixel image, and heterogeneous tumor patterns. This paper proposes an attention-enhanced UNet architecture for segmenting cancerous (Gleason score 3) regions in WSIs of prostate gland. The proposed method combines attention gates with subsequent dropout regularization at skip connections, specifically designed to handle subtle glandular malignancies and maintain consistent feature extraction across WSIs. The proposed model achieves an average Dice score of 0.8208 in the validation dataset. The experimental results demonstrates the superiority of the proposed model over other UNet-based models.