A Review on Graph Theory in Deep Learning

Graph theory has gained popularity as a flexible tool in machine learning to capture complex correlations between objects in non-Euclidean structured data. Graph Neural Networks (GNNs) have proved to be effective deep learning methods to learn node and graph-level representations using message passing, attention and convolution applied to graph structures. This review classifies more than 39 GNN variants into five structural families: Graph Convolutional Networks (GCNs), Graph Attention Networks (GATs), Graph Sample and Aggregate (SAGE), Graph Isomorphism Networks (GINs) and attention-augmented or hybrid models. In addition to the recent models AA-HGNN, DySAT, EGNN, DyHAN, Hyper-GNN, and MGH, which are built to include heterogeneity, scalability, dynamic graphs, and geometric invariance. These models acknowledge the increasing diversity of GNN architectures adapted to real-life requirements in such areas as recommendation systems, bioinformatics, and social networks. Additionally, the review lists important challenges to GNN development, which included overfitting, explainability, and multi-modal learning, and provides directions in the future. To illustrate the development and use of GNN models in numerous directions, this paper provides a graph-centric structure through which researchers can learn how the GNN models are evolving and how it is used in several directions.