The impact of an LLM-based educational agent on learning achievement, cognitive dynamics, and student perceptions in computer science education

While Large Language Models (LLMs) are increasingly integrated into educational settings, the transition from passive chatbot to autonomous LLM-based agents—characterized by tool use, memory retention, and goal-directed reasoning—remains underexplored. To address this gap, this study designed a domain-specific educational agent (DBagent) for an undergraduate database course, and examined its impact on learning achievement, cognitive engagement patterns, and student perceptions. A large-scale quasi-experiment was conducted with 313 sophomore students across four authentic classes (three experimental and one control) over a four-week period. We employed statistical analyses to examine learning achievement and the distribution of cognitive engagement levels, lag sequential analysis (LSA) to analyze cognitive behavior sequences from interaction logs, and structural equation modeling (SEM) to interpret survey data. Findings indicate that (1) the agent-enriched environment significantly improved learning achievement compared with traditional instruction; (2) LSA revealed distinct cognitive trajectories: while student-instructor interactions fostered balanced cognitive levels, student-agent interactions were characterized by high-frequency engagement driven by a specific “Query-Evaluation-Query” verification loop; and (3) SEM analysis confirmed that learners’ positive perceptions of the educational agent promoted sustained engagement through the mediating role of satisfaction. Taken together, the findings indicate that educational agents can improve learning achievement, foster distinctive cognitive engagement patterns, and sustain learner participation through positive user perceptions. These findings provide empirical evidence for the pedagogical value of educational agents and offer practical implications for the design of GenAI-enriched learning environments.