A Method for Reasoning Attribution Identification and Instructional Early Warning in Medical Microbiology Education: A Randomized Controlled Trial

Abstract Background: The widespread adoption of generative artificial intelligence in medical education has made the phenomenon of "AI borrowing" in the classroom a direct threat to the evaluation of learning authenticity. Pathogen reasoning tasks in medical microbiology inherently have a strict dependency structure. However, existing AIGC text detectors remain confined to surface features, neural cognitive diagnosis systems stop at knowledge-component-level scores, and educational early warning tools provide only outcome labels. None of these approaches fills the methodological gap between node-level attribution identification and actions that teachers can execute. Methods: This study proposes EduAttr, which introduces three principled innovations. The reasoning commitment consistency test based on counterfactual premise perturbation introduces the intervention semantics of causal inference into attribution identification. The identifiable path signature based on the dependency partial order violation rate uses the downward closure property of knowledge space theory and forms a complementary coupling with the counterfactual probe. The instructional early warning node ranking mechanism integrates dual-channel signals with dependency depth weights and outputs a Top-K list of follow-up questions that teachers can execute. The system was evaluated in a two-arm randomized controlled trial at two medical colleges in China between September 2024 and June 2025, with 240 second-year clinical medicine students in eight intact classes allocated 1:1 to supervised writing without generative AI or to AI-permitted writing whose interaction transcripts anchored node-level attribution labels. The primary outcome was the node-level F 1 score. Results: Across 1,200 reports and 22,362 annotated nodes, EduAttr achieves a node-level F 1 of 0.843, exceeding the strongest surface feature baseline Ghostbuster by 13.0 percentage points. Under severe rewriting attacks, this gap further expands to 29.5 percentage points. Agreement with the adjudicated labels reaches a Cohen's kappa of 0.762, which is statistically indistinguishable from the interteacher agreement of 0.741, the teacher adoption rate is 87.5%, and grading time is reduced by 42.7%. Conclusions: EduAttr advances identification accuracy, robustness, and instructional operability, providing a technical pathway that can be embedded into classroom follow-up questioning workflows in medical education. Trial registration: Chinese Clinical Trial Registry, ChiCTR2600118749. Registered 2 October 2026 (retrospectively registered).

Authors

Institutions

Publication Details

Journal
Research Square
Published
2026-11-16
DOI
https://doi.org/10.21203/rs.3.rs-11116981/v1
Primary Topic
Academic integrity and plagiarism
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

A Method for Reasoning Attribution Identification and Instructional Early Warning in Medical Microbiology Education: A Randomized Controlled Trial

Fang Wang, Lu Hou, Yue Shi, Silin Zhang
5 links
Research Square
Academic integrity and plagiarism
preprint

A Method for Reasoning Attribution Identification and Instructional Early Warning in Medical Microbiology Education: A Randomized Controlled Trial

Fang Wang, Lu Hou, Yue Shi, Silin Zhang
preprint en

Abstract

Abstract Background: The widespread adoption of generative artificial intelligence in medical education has made the phenomenon of "AI borrowing" in the classroom a direct threat to the evaluation of learning authenticity. Pathogen reasoning tasks in medical microbiology inherently have a strict dependency structure. However, existing AIGC text detectors remain confined to surface features, neural cognitive diagnosis systems stop at knowledge-component-level scores, and educational early warning tools provide only outcome labels. None of these approaches fills the methodological gap between node-level attribution identification and actions that teachers can execute. Methods: This study proposes EduAttr, which introduces three principled innovations. The reasoning commitment consistency test based on counterfactual premise perturbation introduces the intervention semantics of causal inference into attribution identification. The identifiable path signature based on the dependency partial order violation rate uses the downward closure property of knowledge space theory and forms a complementary coupling with the counterfactual probe. The instructional early warning node ranking mechanism integrates dual-channel signals with dependency depth weights and outputs a Top-K list of follow-up questions that teachers can execute. The system was evaluated in a two-arm randomized controlled trial at two medical colleges in China between September 2024 and June 2025, with 240 second-year clinical medicine students in eight intact classes allocated 1:1 to supervised writing without generative AI or to AI-permitted writing whose interaction transcripts anchored node-level attribution labels. The primary outcome was the node-level F 1 score. Results: Across 1,200 reports and 22,362 annotated nodes, EduAttr achieves a node-level F 1 of 0.843, exceeding the strongest surface feature baseline Ghostbuster by 13.0 percentage points. Under severe rewriting attacks, this gap further expands to 29.5 percentage points. Agreement with the adjudicated labels reaches a Cohen's kappa of 0.762, which is statistically indistinguishable from the interteacher agreement of 0.741, the teacher adoption rate is 87.5%, and grading time is reduced by 42.7%. Conclusions: EduAttr advances identification accuracy, robustness, and instructional operability, providing a technical pathway that can be embedded into classroom follow-up questioning workflows in medical education. Trial registration: Chinese Clinical Trial Registry, ChiCTR2600118749. Registered 2 October 2026 (retrospectively registered).

Research Square
Changchun University of Chinese Medicine (CN)
Quality education
Academic integrity and plagiarism
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon