MS-GATOR: multi-scale graph attention with topological reasoning for segmentation and classification of prostate cancer based on Gleason scores using histopathology images

Prostate cancer is one of the prevalent and potentially fatal malignancies, affecting men globally. Accurate segmentation and classification of prostate cancer in histopathology images remain challenging due to complex tissue architecture, staining variability, and high inter-observer differences in manual diagnosis. To address these challenges, this research proposes a Multi-Scale Graph Attention with Topological Reasoning (MS-GATOR) for segmentation and classification of prostate cancer based on Gleason Scores (GS). The MS-GATOR incorporates topological reasoning with hierarchical biological structures by constructing a graph-based representation among three spatial scales such as nuclei, glands, and tissues using Delaunay triangulation to capture both geometric and photometric relationships. Graph attention layers combined with cross-level attention enable efficient intra and inter-scale feature fusion. The MS-GATOR also improves interpretability by visualizing attention weights, while aligning with pathologists' evaluation. The MS-GATOR is validated on publicly available datasets such as PANDA, SICAPv2, and real-time clinical data. It achieved superior performance with dice scores of 99.58% and 98.67%, as well as classification accuracies of 99.95% and 98.86% on PANDA and SICAPv2, respectively. Additionally, morphological and statistical analyses demonstrate robustness across tumor grades and patient subgroups. Compared to traditional CNN and GNN approaches, MS-GATOR provides better accuracy, generalization, and clinical significance. These results highlight MS-GATOR's potential as a scalable, explainable, and effective solution for pathology, thereby supporting consistent and reasonable prostate cancer classification.