Attentional Neural Integral Equation for Temporal Knowledge Graph Forecasting

Temporal Knowledge Graph Forecasting (TKGF) aims to forecast the missing entities or relations at a specific timestamp when only the historical information is observed. It is crucial to accurately identify the historical information of complex temporal relational graphs related to the query. Existing works, e.g., TANGO, have exploited the Neural Ordinary Differential Equation (NODE) to TKGF. However, TANGO encounters two limitations. First, TANGO observes historical facts with only one timestamp at each step, leading to a long-term forgetting problem. Second, TANGO gives the same weight to the entire history graph, including facts that are not relevant to the query. To tackle the above limitations, this paper utilizes Attentional Neural Integral Equation for TKGF (tIE), enabling the global interaction between query-related historical graph sequences. To achieve this, we employ the Relational Graph Convolutional Network and Fourier-type Transformer to model the graph structure and temporal evolution of TKG. The Iterative Integral Equation Solver is exploited to enhance the accuracy and robustness of numerical solutions. The proposed method outperforms baseline models regarding several metrics and inference speed on four benchmark datasets, especially on the long horizontal link forecasting task with irregular time intervals.