Graph retrieval-augmented generation for legal academic viewpoint question answering: knowledge-graph retrieval, evaluation, and hallucination analysis

Large language models (LLMs) have a common problem of hallucination in legal academic viewpoint question answering, which limits its reliability in legal research assistance. To alleviate this problem, this study uses Chinese core legal journal literature as the data source to construct a graph retrieval augmented generation (GraphRAG) framework for legal academic viewpoint question answering in this field and systematically compares the performance of two strategies, namely Vector and Cypher retrieval, on three LLMs. The experimental results show that with the Cypher retrieval strategy under the neo4j_graphrag framework, the unanswerable rate of gpt-4-turbo decreased to 3.41% and the BERTScore-F1_avg reached 80.58%, which provides preliminary evidence for the effectiveness of structured knowledge graph retrieval in improving the answer quality. Through example analysis and quantification, this study summarized five GraphRAG-specific hallucination types, revealing that GraphRAG alleviates traditional knowledge-deficiency hallucination while shifting the source of hallucinations from the blind fabrication of the model to the collaborative retrieval and generation process, and that certain differences exist in the distribution of hallucination types under different retrieval strategies. This study focuses on the development and optimization of a legal academic question and answer system under the theme “Legal Regulation of AI Training Data,” within a known and closed corpus, and provides a reference for the study of possible hallucination generated by the GraphRAG framework in this context.

Authors

Institutions

Publication Details

Journal
Frontiers in Artificial Intelligence
Published
2026-11-12
DOI
https://doi.org/10.3389/frai.2026.1922684
Primary Topic
Topic Modeling
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Graph retrieval-augmented generation for legal academic viewpoint question answering: knowledge-graph retrieval, evaluation, and hallucination analysis

Jianhao Tang, Jia Liu
Frontiers in Artificial Intelligence
Topic Modeling
article

Graph retrieval-augmented generation for legal academic viewpoint question answering: knowledge-graph retrieval, evaluation, and hallucination analysis

Jianhao Tang, Jia Liu
article en

Abstract

Large language models (LLMs) have a common problem of hallucination in legal academic viewpoint question answering, which limits its reliability in legal research assistance. To alleviate this problem, this study uses Chinese core legal journal literature as the data source to construct a graph retrieval augmented generation (GraphRAG) framework for legal academic viewpoint question answering in this field and systematically compares the performance of two strategies, namely Vector and Cypher retrieval, on three LLMs. The experimental results show that with the Cypher retrieval strategy under the neo4j_graphrag framework, the unanswerable rate of gpt-4-turbo decreased to 3.41% and the BERTScore-F1_avg reached 80.58%, which provides preliminary evidence for the effectiveness of structured knowledge graph retrieval in improving the answer quality. Through example analysis and quantification, this study summarized five GraphRAG-specific hallucination types, revealing that GraphRAG alleviates traditional knowledge-deficiency hallucination while shifting the source of hallucinations from the blind fabrication of the model to the collaborative retrieval and generation process, and that certain differences exist in the distribution of hallucination types under different retrieval strategies. This study focuses on the development and optimization of a legal academic question and answer system under the theme “Legal Regulation of AI Training Data,” within a known and closed corpus, and provides a reference for the study of possible hallucination generated by the GraphRAG framework in this context.

Frontiers in Artificial IntelligenceVol. 9
Beijing Institute of Big Data Research (CN), Huazhong University of Science and Technology (CN)
Peace, justice, and strong institutions
Topic Modeling
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon