Beyond Node Condensation: Learning Tiny Graphs via Joint Graph Condensation

Training graph neural networks (GNNs) on large-scale graphs is challenging due to the high computational cost of numerous nodes and high-dimensional features. Existing studies tackle this problem only by reducing the number of nodes in the graph. However, the resulting condensed graph data can still be cumbersome. To address this issue, we introduce a novel approach called joint condensation, and propose a novel framework, TinyGraph, which condenses both features and nodes simultaneously while preserving the critical feature and graph information of the original graph. Specifically, TinyGraph formulates the problem as matching the gradients of the GNN weights trained on the condensed graph to the gradients derived from training on the original graph. The condensed graph obtained by minimizing the matching loss along the training trajectory can henceforth retain critical information in the original graph. Extensive experiments were carried out to demonstrate the effectiveness of the proposed TinyGraph. For example, a GNN trained with TinyGraph retains 98.5% and 97.5% of the original test accuracy on the Cora and Citeseer datasets, respectively, while significantly reducing the number of nodes by 97.4% and 98.2%, and the number of features by 90.0% on both datasets.