Relieving the Over-Smoothing Problem using Graph Disentanglement with Latent Bottlenecks

Graph Convolutional Networks (GCNs) have been widely adopted to learn structured data that denotes structural connections between entities. However, over-smoothing has become a common phenomenon in GCNs, resulting in node representations in different classes are indistinguishable. Inspired by recent works, graph disentanglement, which can learn disentangled latent factors behind the input data and generate latent representations corresponding to latent factors, is adopted to relieve over-smoothing. Although graph disentanglement is promising, it still faces some challenges. (1) How to obtain disentangled latent representations useful for both latent factors and downstream tasks? (2) How to enhance the diversity and independence of latent representations? In this paper, we propose a novel GCN model based on Graph Disentanglement with Latent Bottlenecks (DBGCN) to boost the diversity of node representations and relieve the over-smoothing problem. To learn disentangled node representations that are beneficial to both latent factors and downstream tasks, we propose to identify latent bottlenecks using the information bottleneck, which offers a critical principle for good representation learning. To boost the diversity of latent representations and the independence between them, we design a generative model to search for specific latent bottlenecks that constrain information flow from the input using specific latent factors, and to derive distributions of latent representations. Experimental results on various real-world graphs demonstrate that the proposed model can relieve the over-smoothing problem effectively and achieve significant performance gains. It also suggests that enhancing the diversity of latent representations is helpful to relieve over-smoothing.