WikiRAG: Revisiting Wikidata KGC Datasets with Community Updates and Retrieval-Augmented Generation

Link prediction is an important task for knowledge graph completion and curation, and it has received significant attention from the research community. However, researchers often train and evaluate new models on small or outdated datasets that do not reflect the current state of knowledge, thereby disregarding new information and the rich textual content often linked to knowledge graphs. As a result, many opportunities to leverage these dimensions are missed. We introduce WikiRAG, a framework for knowledge completion and evaluation derived from Wikidata and Wikipedia, which enables research integrating retrieval techniques and large language models. Our framework combines the following contributions: (i) We revisit the Wikidata5M dataset by updating it to reflect the current state of Wikidata and providing automated tools for its periodic maintenance. (ii) We enrich the dataset with long-form textual content sourced from Wikipedia, enabling research that goes beyond traditional graph structures and shallow text methods toward dense retrieval techniques. (iii) We propose a simple yet effective baseline that leverages retrieval-augmented generation, demonstrating the utility of the dataset and integrating language model capabilities for link prediction. The revised dataset, coined Wikidata5M-RE, shows that the original graph grew by roughly 50% in the number of edges, while 10% of the edges have been removed. A comparative analysis of classic methods demonstrates that these changes can impact downstream task evaluation. Finally, our evaluation of WikiRAG's KGC method shows an improvement of up to 9% in link prediction accuracy over state-of-the-art baselines, setting the stage for a new avenue in knowledge completion that uses deep information extraction. The source code, data, and other artifacts have been made available on the project website: https://github.com/colab-nyuad/WikiRAG