GRP-010 Autonomous Agents in the Loop: Strengthening Educational Recommenders with Computer Use Agents

Pedagogical Design Patterns (PDPs) serve as reusable, research-informed strategies that support effective teaching, yet their discoverability remains a major hurdle for educators. In this work, we extend the PDPR (Personalized Dynamic Practice and Reflection) system with a Retrieval-Augmented Generation (RAG) framework powered by a fine-tuned large language model (LLM) to deliver context-aware PDP recommendations. A key innovation in our proposed system is the integration of a Computer-Using Agent (CUA), which acts as a fallback mechanism when the internal knowledge base lacks sufficient coverage or yields low-confidence responses. This agent autonomously interacts with a live desktop environment—using browser automation, mouse control, keyboard input, and screenshots—to search the web, download educational resources, and summarize external content relevant to the user's input query. Through a dynamic agentic loop, the CUA bridges the gap between static knowledge and real-time discovery, expanding the system's ability to provide grounded, practical instructional support. We evaluate our framework using the RAG Triad methodology and qualitative feedback from educators, demonstrating both high relevance and adaptability of the recommendations. This hybrid architecture not only strengthens pedagogical support for educators but also pushes the boundary of AI-driven educational tools by blending structured retrieval with autonomous information-seeking Agents.