A Survey on Self-Supervised Graph Foundation Models: Knowledge-Based Perspective

The field of graph foundation models (GFMs) has seen a dramatic rise in interest in recent years. Their powerful generalization ability is believed to be endowed by self-supervised pre-training and downstream tuning techniques. There is a wide variety of knowledge patterns embedded in the graph data, such as node properties and clusters, which are crucial for learning generalized representations for GFMs. We present a comprehensive survey of self-supervised GFMs from a novel knowledge-based perspective. Our main contribution is a knowledge-based taxonomy that categorizes self-supervised graph models by the specific graph knowledge utilized: microscopic (nodes, links, etc.), mesoscopic (context, clusters, etc.), and macroscopic (global structure, manifolds, etc.). It covers a total of 9 knowledge categories and 300 references for self-supervised pre-training as well as various downstream tuning strategies. Such a knowledge-based taxonomy allows us to more clearly re-examine potential GFM architectures, including large language models (LLMs), as well as provide deeper insights for constructing future GFMs.