Ontology-Driven Archival Knowledge Graph Construction Leveraging Large Language Models

The fusion of artificial intelligence with archival studies, particularly through knowledge graphs, has revolutionized the ways in which archival materials are organized, retrieved, and analyzed. This shift has been fueled by the rapid advancement of automatic NLP techniques, vital for extracting relevant information from textual archives. The potential effectiveness of Large Language Models (LLMs), e.g., GPT-4, in constructing knowledge graphs is highlighted by their impressive performance across various NLP tasks. In our research, we explore the utility of multi-platform LLMs in constructing knowledge graphs for archival resources. To this end, we developed a domain-specific ontology as a benchmark to assess the effectiveness of these models in tasks. Our methodology included the creation of knowledge graphs through manual annotation of archival texts, followed by the application of LLMs to automate this graph construction. This approach allowed us to compare the manually curated graphs, which served as our ground truth, with those generated by the LLMs. Our evaluation, based on precision, recall, and F1 scores, shows that GPT-4 consistently surpasses other models in performance. These results highlight the considerable potential of LLMs in streamlining the creation of knowledge graphs specifically designed for archival contexts.