A Scaled Mask Attention-based Hybrid model for Survival Prediction

Whole Slide Images (WSIs) are widely used in medical practice, but predicting survival from histopathological images is particularly challenging due to the vast scale and intricate structure of WSIs. Traditional Convolutional Neural Network (CNN)-based methods, which divide WSIs into multiple patches for analysis, often struggle with the heterogeneity of WSIs. They often lose crucial feature information during patch sampling and fail to fully capture spatial, contextual, and hierarchical interactions. Moreover, many current methods do not adequately consider correlations between patches and struggle to adapt to the extensive dimensions of WSIs. To overcome these limitations, we introduce a novel hybrid model that integrates CNNs, transformers, and Graph Convolutional Networks (GCNs) for end-to-end survival prediction. Firstly, the model employs ResNet-50 and a Learnable Linear Projection (LLP) to extract the complex and high-dimensional features. Subsequently, the Scaled Masked Attention (SMA) proposes to focus on high-attention regions within informative features. Additionally, cosine similarity (CS) emphasizes characteristics within structurally similar features. Finally, the proposed model utilizes GCNs to effectively aggregate the extracted features, capturing the complex correlations between them to enhance survival prediction. To evaluate the performance of the proposed method, we use five public datasets. Extensive experiments and ablation studies demonstrate that our model outperforms existing methods in terms of C-index.