An Evidence-Constrained RAG Learning Support System for Interdisciplinary Smart Water Engineering Education: Multi-Source Knowledge Governance and Reproducible Technical Evaluation

Smart water engineering is inherently interdisciplinary, requiring students to synthesize hydrological and hydraulic principles, observational data, domain-specific models, engineering standards, and computational tools to support evidence-based reasoning and decision-making. Yet, the knowledge required for such learning is fragmented across textbooks, technical standards, model manuals, and software documentation, while general-purpose large language models (LLMs) provide limited guarantees regarding the relevance, provenance, and traceability of generated information. To address this challenge, this study adopted a design science research approach to develop and technically evaluate an evidence-constrained retrieval-augmented generation (RAG) framework for interdisciplinary smart water engineering education. In this article, trustworthiness is used in a deliberately operational sense: provenance and version traceability, evidence support, citation-scope control, and boundary-aware refusal. It does not denote demonstrated user trust, professional correctness, or learning improvement. The framework integrates multi-source knowledge governance, hybrid retrieval, reranking, citation constraints, evidence-gap detection, and structured refusal into an auditable evidence-support pipeline. The frozen knowledge base comprised 159 registered sources, 17,295 retrievable chunks, and 826 aligned Chinese–English technical terms. Evaluation on a frozen 150-question benchmark showed that the full framework achieved a Recall@5 of 0.8256 and an nDCG@10 of 0.7717, exceeding the lexical-retrieval baseline by 0.3233 and 0.3081, respectively. On a separate 180-question refusal benchmark, it achieved an F1 score of 0.8451 while maintaining an acceptance rate of 0.9267 for answerable questions. Under a 20-question cross-domain holdout, evidence-gap cascading increased Recall@5 from 0.5750 to 0.7000; this small-sample result is exploratory and does not establish generalizable cross-domain performance. The author-retained frozen records, analysis scripts, prompt versions, and hashes can support independent verification when legally shareable materials are provided upon reasonable request, whereas calls to hosted model services and access to restricted or third-party source documents cannot be reproduced exactly without the same provider state and permissions. No students or domain experts participated in the evaluation. The findings therefore establish technical feasibility and evidence-auditability, not educational effectiveness or expert-validated professional correctness.

Authors

Institutions

Publication Details

Journal
Applied Sciences
Published
2026-10-05
DOI
https://doi.org/10.3390/app16199874
Primary Topic
Artificial Intelligence in Education
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

An Evidence-Constrained RAG Learning Support System for Interdisciplinary Smart Water Engineering Education: Multi-Source Knowledge Governance and Reproducible Technical Evaluation

Yuan Yuan, Yuhui Cai, Ming Huang, Xianglong Wei et al.
Applied Sciences
Artificial Intelligence in Education
article

An Evidence-Constrained RAG Learning Support System for Interdisciplinary Smart Water Engineering Education: Multi-Source Knowledge Governance and Reproducible Technical Evaluation

Yuan Yuan, Yuhui Cai, Ming Huang, Xianglong Wei, Yu Chen, Jiarun Huang, Jing Liu
article en

Abstract

Smart water engineering is inherently interdisciplinary, requiring students to synthesize hydrological and hydraulic principles, observational data, domain-specific models, engineering standards, and computational tools to support evidence-based reasoning and decision-making. Yet, the knowledge required for such learning is fragmented across textbooks, technical standards, model manuals, and software documentation, while general-purpose large language models (LLMs) provide limited guarantees regarding the relevance, provenance, and traceability of generated information. To address this challenge, this study adopted a design science research approach to develop and technically evaluate an evidence-constrained retrieval-augmented generation (RAG) framework for interdisciplinary smart water engineering education. In this article, trustworthiness is used in a deliberately operational sense: provenance and version traceability, evidence support, citation-scope control, and boundary-aware refusal. It does not denote demonstrated user trust, professional correctness, or learning improvement. The framework integrates multi-source knowledge governance, hybrid retrieval, reranking, citation constraints, evidence-gap detection, and structured refusal into an auditable evidence-support pipeline. The frozen knowledge base comprised 159 registered sources, 17,295 retrievable chunks, and 826 aligned Chinese–English technical terms. Evaluation on a frozen 150-question benchmark showed that the full framework achieved a Recall@5 of 0.8256 and an nDCG@10 of 0.7717, exceeding the lexical-retrieval baseline by 0.3233 and 0.3081, respectively. On a separate 180-question refusal benchmark, it achieved an F1 score of 0.8451 while maintaining an acceptance rate of 0.9267 for answerable questions. Under a 20-question cross-domain holdout, evidence-gap cascading increased Recall@5 from 0.5750 to 0.7000; this small-sample result is exploratory and does not establish generalizable cross-domain performance. The author-retained frozen records, analysis scripts, prompt versions, and hashes can support independent verification when legally shareable materials are provided upon reasonable request, whereas calls to hosted model services and access to restricted or third-party source documents cannot be reproduced exactly without the same provider state and permissions. No students or domain experts participated in the evaluation. The findings therefore establish technical feasibility and evidence-auditability, not educational effectiveness or expert-validated professional correctness.

Applied SciencesVol. 16(19)
Hohai University (CN), Nanjing Hydraulic Research Institute (CN), Ministry of Water Resources of the People's Republic of China (CN)
Openalex Percentile: Top 5%
Artificial Intelligence in Education
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.