GraphRAG-Based Generation of Mathematics Assessments Using Prerequisite Knowledge Graphs

Large language models (LLMs) provide new opportunities for the automatic generation of educational assessments. However, their application in mathematics remains challenging because they do not explicitly represent prerequisite knowledge and may generate mathematically incorrect content. To address these limitations, this study proposes a GraphRAG-based architecture for generating multiple-choice mathematics assessments. The architecture integrates a directed acyclic knowledge graph (DAG), semantic embeddings, hybrid retrieval, LLM-based generation, and independent mathematical validation using SymPy. The knowledge graph represents prerequisite relationships among mathematical concepts, allowing the system to retrieve structured information to guide question and distractor generation. The evaluation included 154 questions covering basic fraction operations and 462 distractors, of which 53.2% showed direct correspondence with prerequisite knowledge and 30.7% showed indirect correspondence. Mathematical validation accepted 137 questions (89.0%) without correction, while 17 (11.0%) required correction and subsequently passed revalidation. In addition, a controlled comparison involving 80 paired generation requests examined conventional RAG and GraphRAG under comparable generation conditions and the same maximum context limit. GraphRAG produced a higher proportion of distractors with identifiable prerequisite relationships than conventional RAG (91% versus 77%). A complementary evaluation by 24 mathematics teachers using a five-point Likert scale showed favorable perceptions across five dimensions, with coherence receiving the highest mean score (M = 4.68) and usefulness for teachers the lowest (M = 4.36). Overall, the findings provide preliminary evidence of the technical feasibility of the proposed architecture and suggest that the complete GraphRAG retrieval configuration is associated with stronger prerequisite–distractor correspondence. However, the comparison does not isolate the contribution of graph relationships from possible differences in retrieved context, and the study does not establish diagnostic validity or improvements in student learning outcomes.

Authors

Institutions

Publication Details

Journal
Applied Sciences
Published
2026-11-12
DOI
https://doi.org/10.3390/app162211202
Primary Topic
Artificial Intelligence in Education
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

GraphRAG-Based Generation of Mathematics Assessments Using Prerequisite Knowledge Graphs

Juan Arturo Pérez-Cebreros, Francisco Cervantes, Joahan Israel Hernández-Granados
Applied Sciences
Artificial Intelligence in Education
article

GraphRAG-Based Generation of Mathematics Assessments Using Prerequisite Knowledge Graphs

Juan Arturo Pérez-Cebreros, Francisco Cervantes, Joahan Israel Hernández-Granados
article en

Abstract

Large language models (LLMs) provide new opportunities for the automatic generation of educational assessments. However, their application in mathematics remains challenging because they do not explicitly represent prerequisite knowledge and may generate mathematically incorrect content. To address these limitations, this study proposes a GraphRAG-based architecture for generating multiple-choice mathematics assessments. The architecture integrates a directed acyclic knowledge graph (DAG), semantic embeddings, hybrid retrieval, LLM-based generation, and independent mathematical validation using SymPy. The knowledge graph represents prerequisite relationships among mathematical concepts, allowing the system to retrieve structured information to guide question and distractor generation. The evaluation included 154 questions covering basic fraction operations and 462 distractors, of which 53.2% showed direct correspondence with prerequisite knowledge and 30.7% showed indirect correspondence. Mathematical validation accepted 137 questions (89.0%) without correction, while 17 (11.0%) required correction and subsequently passed revalidation. In addition, a controlled comparison involving 80 paired generation requests examined conventional RAG and GraphRAG under comparable generation conditions and the same maximum context limit. GraphRAG produced a higher proportion of distractors with identifiable prerequisite relationships than conventional RAG (91% versus 77%). A complementary evaluation by 24 mathematics teachers using a five-point Likert scale showed favorable perceptions across five dimensions, with coherence receiving the highest mean score (M = 4.68) and usefulness for teachers the lowest (M = 4.36). Overall, the findings provide preliminary evidence of the technical feasibility of the proposed architecture and suggest that the complete GraphRAG retrieval configuration is associated with stronger prerequisite–distractor correspondence. However, the comparison does not isolate the contribution of graph relationships from possible differences in retrieved context, and the study does not establish diagnostic validity or improvements in student learning outcomes.

Applied SciencesVol. 16(22)
Instituto Tecnológico y de Estudios Superiores de Occidente (MX), Universidad Autónoma Metropolitana (MX), Universidad Internacional De La Rioja (ES), Instituto Politécnico Nacional (MX)
Quality education
Artificial Intelligence in Education
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon