DeepSGE: predicting spatial gene expression using residual network with efficient channel attention and dynamic graph attention network

Predicting spatially resolved gene expression from histological images is a crucial step in analyzing spatial transcriptomics (ST) data. Recently, deep learning (DL)-based methods have been proposed to predict spatial gene expression and achieved impressive performance. However, there are several limitations to these methods. First, they cannot effectively capture the complex texture structure and deep features of histological images. Second, they assign fixed weights to neighbor nodes, making it difficult to adaptively emphasize important nodes, thus limiting their performance in capturing complex graph structures and long-distance dependencies. Finally, traditional static attention mechanism prevents the model from fitting the training data. We propose a novel DL-based model, DeepSGE, based on residual network with efficient channel attention (ECA-ResNet), Vision Transformer, and dynamic graph attention network (Dynamic-GAT) for predicting spatially resolved gene expression from histological images. DeepSGE extracts spot image features through ECA-ResNet, then models global and local dependencies between spots through Vision Transformer and Dynamic-GAT to fully explore spatial dependencies in histology. We compared the performance of DeepSGE with four state-of-the-art methods on three ST datasets (one in mouse and two in human). Experimental results show that DeepSGE outperforms these competing methods. DeepSGE outperforms competing methods in spatial gene expression prediction and spatial clustering, and can reveal the intrinsic connection between cellular molecular characteristics and tissue structure. The source code for DeepSGE is available at https://github.com/nathanyl/DeepSGE.