Multi-level contrastive learning for exploring graphs with higher-order dependencies in sequential data

Graph embedding, which maps nodes into a low-dimensional space while preserving their proximities based on graph topology, has gained significant attention. Traditional methods often assume the Markov property, thus overlooking higher-order dependencies. Additionally, most Graph Neural Network-based methods require abundant labeled data, which is usually scarce and costly. To address these challenges, we introduce Higher-Order Contrastive Graph Embedding (HOCGE), a novel self-supervised graph embedding framework that simultaneously models first-order structures and higher-order dependencies. Specifically, HOCGE learns node representations by constructing higher-order networks from sequential data, capturing complex node relationships through advanced random walks. By designing a multi-level contrastive learning mechanism, HOCGE narrows the distance between similar nodes across both first-order and higher-order networks in the embedding space. This self-supervised method alleviates the issue of data scarcity by utilizing the intrinsic structure of the data, reducing the dependence on labeled data. Comprehensive experiments on various benchmarks, including link prediction, node classification, and visualization, demonstrate HOCGE’s superior performance, robustness, and generalizability.