InfoMLP: Unlocking the Potential of MLPs for Semi-Supervised Learning with Structured Data

We introduce InfoMLP, an innovative model structured like a Multilayer Perceptron (MLP) for semi-supervised classification of structured data, e.g., graphs. InfoMLP was inspired by our observation that overlapping information between node features and the structure between data points significantly influences the performance gap between feature-only MLPs and advanced graph-based semi-supervised methods, e.g., GNNs. To quantify the overlapping information, we first introduce a tractable metric to quantify the mutual information between node features and graph structure. Based on this, we propose InfoMLP, which seeks to maximize the mutual information between node embeddings derived from the MLP and the structure information. Our info-max objective is split into two sub-objectives: the first is a non-parametric preprocessing step aiming to find the optimal graph-augmented node feature matrix that captures the maximal information about the graph structure, while the second sub-objective is to maximize the mutual information between node embeddings generated from the original node features and those from the graph-augmented node features. Since the message-passing operation is integrated into the preprocessing step, requiring only a single execution per dataset, InfoMLP maintains the same efficiency as a vanilla MLP during both training and testing. We validate the efficacy of our design through experiments on real-world datasets of varying scales supplemented by comprehensive ablation studies. Our results corroborate our analysis and demonstrate the effectiveness of our novel approach.