Graph Curvature Flow-Based Masked Attention

Graph\nneural networks (GNNs) have revolutionized drug discovery\nin chemistry and biology, enhancing efficiency and reducing resource\ndemands. However, classical GNNs often struggle to capture long-range\ndependencies due to challenges like oversmoothing and oversquashing.\nGraph Transformers address these issues by employing global self-attention\nmechanisms that allow direct information exchange between any pair\nof nodes, enabling the modeling of long-range interactions. Despite\nthis, Graph Transformers often face difficulties in capturing the\nnuanced structural information on graphs. To overcome these challenges,\nwe introduce the CurvFlow-Transformer, a novel graph Transformer model\nincorporating a curvature flow-based masked attention mechanism. By\nleveraging a topologically enhanced mask matrix, the attention layer\ncan effectively detect subtle structural differences within graphs,\nbalancing the focus between global mutual information and local structural\ndetails of molecules. The CurvFlow-Transformer demonstrates superior\nperformance on the MoleculeNet data set, surpassing several state-of-the-art\nmodels across various tasks. Moreover, the model provides unique insights\ninto the relationship between molecular structure and chemical properties\nby analyzing the attention heat coefficients of individual atoms.