Representation Learning Based on Ordinary Differential Equations for Dynamic Networks

Representation learning on networks, mapping the network into a low-dimensional vector space, has received signification attention recently due to its widespread application in graph data mining tasks. With the success of representation learning in static networks, we push further for practical scenarios of dynamic networks. Existing methods model the dynamic network by dividing the dynamic network into sequences of network snapshots, see each network snapshot as a static network, and utilize the dynamic evolution between snapshots, they capture the discrete dynamic evolution of dynamic networks. However, a dynamic network continuously evolves over time. Capturing the continuously dynamic evolution of dynamic networks is important for dynamic network representation. In this article, we regard a dynamic network as a dynamic system, use the ordinary differential equation (ODE) to model the dynamic evolution of dynamic networks, and integrate the ODE over continuous-time to capture continuously dynamic evolution of dynamic networks; and design a new encoder-decoder model for dynamic networks representation. We improve the gated recurrent unit (GRU) module (only capturing the discrete dynamic evolution of the dynamic network and structure information) by combining an ODE and a GRU. The improved GRU as the encoder can learn the continuously dynamic evolution, and structure information of the dynamic network, where the ODE parameterized by a graph neural network models the continuously dynamic evolution of each network snapshot. Use the ODE and Inner-Productor as the decoder, where the ODE is integrated over continuous-time to learn the continuous dynamic evolution of the latent representation of the whole dynamic network, and the Inner-Productor reconstructs the topological structure of each snapshot by doing the inner-product between nodes representation, the reconstructing errors as the objective function of our method. To assess our model, we expand the experiment on several real-world dynamic networks, and results show that our method consistently outperforms existing baselines in three dynamic link prediction tasks; the best is up to 6.54 \( % \) improvement. To our knowledge, our method is the first work using the ODE to capture the continuously dynamic evolution of dynamic networks.