IALab UC at BEA 2025 Shared Task: LLM-Powered Expert Pedagogical Feature Extraction

As AI's presence in educational environments grows, it becomes critical to evaluate how its feedback may impact students' learning processes.Pedagogical theory, with decades of effort into understanding how human instructors give good-quality feedback to students, may provide a rich source of insight into feedback automation.In this paper, we propose a novel architecture based on pedagogical-theory feature extraction from the conversation history and tutor response to predict pedagogical guidance on MRBench.Such features are based on Brookhart's canonical work in pedagogical theory, and extracted by prompting the language model LearnLM.The features are then used to train a random-forest classifier to predict the Track 3: Pedagogical Guidance of the BEA 2025 shared task.Our approach ranked 8 th in the dimension's leaderboard with a test Macro F1-score of 0.54.Our work provides some evidence in support of using pedagogical theory qualitative factors treated separately to provide clearer guidelines on how to improve lowscoring intelligent tutoring systems.Finally, we observed several inconsistencies between pedagogical theory and MRBench's inherent relaxation of the tutoring problem implied in evaluating on a single-conversation basis, calling for the development of more elaborate measures which consider student profiles to serve as true heuristics of AI tutors' usefulness.