Temporal-LLM-KGC: A Diagnostic Study of Temporal Evidence in Large Language Model Knowledge Graph Completion

Large language models (LLMs) have shown promise for temporal knowledge graph completion (temporal KGC), but accuracy alone does not reveal dependence on historical context. We evaluate Normal, No History, Anonymous, and Noisy evidence conditions on ICEWS14 and ICEWS18 with four instruction-tuned LLMs. We also test Temporal Evidence Filtering (TEF), a deterministic, training-free proof-of-concept selector combining relation compatibility, temporal recency, and entity-role consistency under a fixed ten-fact prompt budget. Removing or displacing history lowers accuracy; Anonymous improves it but simultaneously changes the constrained candidate vocabulary, precluding a single-mechanism explanation. TEF has positive descriptive differences in all 24 dataset–model–metric contrasts. Across eight dataset–model pairs, its equal-weight macro Hits@10 gain is +0.031375 (nominal paired-bootstrap 95% confidence interval: [+0.021875, +0.040875]); 10 of the 24 individual contrasts remain significant after Holm correction. Qwen3-4B selector baselines, component ablations, parameter sensitivity, and a disjoint second query sample qualify this result. TEF adds a mean 4.360 and 19.071 ms of CPU selection per query on ICEWS14 and ICEWS18, respectively; it does not compress the Normal prompt or establish an end-to-end speedup. Conclusions concern tail prediction with constrained generated lists on these two datasets, not exhaustive ranking or an identified internal reasoning mechanism.

Authors

Institutions

Publication Details

Journal
Information
Published
2026-11-11
DOI
https://doi.org/10.3390/info17111137
Primary Topic
Advanced Graph Neural Networks
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Temporal-LLM-KGC: A Diagnostic Study of Temporal Evidence in Large Language Model Knowledge Graph Completion

Kuo Zhao, Lingfei Zhang, Chingkwun Lam, Jiaxin Li et al.
Information
Advanced Graph Neural Networks
article

Temporal-LLM-KGC: A Diagnostic Study of Temporal Evidence in Large Language Model Knowledge Graph Completion

Kuo Zhao, Lingfei Zhang, Chingkwun Lam, Jiaxin Li, Rongyu Fan, Xinyi Dai, Dahang Ling, Haotian Lu
article en

Abstract

Large language models (LLMs) have shown promise for temporal knowledge graph completion (temporal KGC), but accuracy alone does not reveal dependence on historical context. We evaluate Normal, No History, Anonymous, and Noisy evidence conditions on ICEWS14 and ICEWS18 with four instruction-tuned LLMs. We also test Temporal Evidence Filtering (TEF), a deterministic, training-free proof-of-concept selector combining relation compatibility, temporal recency, and entity-role consistency under a fixed ten-fact prompt budget. Removing or displacing history lowers accuracy; Anonymous improves it but simultaneously changes the constrained candidate vocabulary, precluding a single-mechanism explanation. TEF has positive descriptive differences in all 24 dataset–model–metric contrasts. Across eight dataset–model pairs, its equal-weight macro Hits@10 gain is +0.031375 (nominal paired-bootstrap 95% confidence interval: [+0.021875, +0.040875]); 10 of the 24 individual contrasts remain significant after Holm correction. Qwen3-4B selector baselines, component ablations, parameter sensitivity, and a disjoint second query sample qualify this result. TEF adds a mean 4.360 and 19.071 ms of CPU selection per query on ICEWS14 and ICEWS18, respectively; it does not compress the Normal prompt or establish an end-to-end speedup. Conclusions concern tail prediction with constrained generated lists on these two datasets, not exhaustive ranking or an identified internal reasoning mechanism.

InformationVol. 17(11)
Jinan University (CN)
Advanced Graph Neural Networks
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon