Educational Knowledge Graph Question Answering: Insights from Benchmarks and LLM-based Solutions

Educational question answering plays an extremely important role in online education all over the world. Among these, Educational Knowledge Graph Question Answering (EKGQA) focuses on extracting knowledge from the educational knowledge base to answer educational questions. Although some work has been done, the scarcity of the real Chinese annotated dataset has restricted the development of EKGQA. Unlike current datasets where the questions are direct, educational questions asked by learners may involve diverse, mixed cognitive expressions and language styles due to differences in personal experiences, cultural backgrounds, and cognitive levels. To address these issues, in this paper, we proposed to benchmark such challenges for EKGQA by establishing a more realistic dataset EDUCEQ. It contains 236.3k questions in 12 categories. Compared with existing datasets, EDUCEQ features two more authentic features - diverse cognitive expressions and various language styles. Additionally, we have conducted an in-depth evaluation of the EDUCEQ dataset and developed a robust LLM-based EKGQA method that established competitive benchmarks for future research. EDUCEQ is the first dataset to portray a more realistic view of educational phenomena from a cognitive and computational linguistics perspective, which is beneficial to educational knowledge graph question answering, AI4Education and also the online education field in general.