Transformers with Temporal Enhanced Attention Over Graphs for Accurate Identification of Pancreatic Cancer

An aggressive from of cancer that rarely shows symptoms in its early stages and hence can be diagnosed at an advanced stage. CT scan or MRI is used to diagnose it in the early stage because early diagnosis gives higher chance of surviving. This work presents an innovative approach to applying deep learning to early pancreatic cancer detection, termed the Transformer-Enhanced Dynamic Spatial-Temporal Graph Network (TED-STGN), which combines Graph Neural Networks (GNNs), Transformer-based self-attention mechanisms, and spatial-temporal analysis to detect the tumour's progression in a series of medical scans. TED-STGN builds a kinetically changing spatial-temporal graph from medical imaging data where each image frame of a video is split into regions that will be the nodes of the graph. The graph attention mechanism controls the dynamic attention distribution for regions which have more apparent changes over time such as tumor area growth. The model uses transformer layers to apply models which capture long-range dependencies between frames and are capable of tracking temporal shifts that mark the initial onset of pancreatic cancers. Furthermore, another enhancement of this model is the self-supervised pre-training phase, which allows better generalization in front of unlabelled data and increases the model's robustness whenever the labelled data are scarce. Based on these metrics the model was tested on several instances with relatively encouraging performance. TED-STGN's assessment ratings were 94.7% accuracy, 95.3% precision and 92.1% recall for the identification of pancreatic cancer.