Emergent Wisdom at BEA 2025 Shared Task: From Lexical Understanding to Reflective Reasoning for Pedagogical Ability Assessment
For the BEA 2025 shared task on pedagogical ability assessment, we introduce LUCERA (Lexical Understanding for Cue Density-Based Escalation and Reflective Assessment), a rubric-grounded evaluation framework for systematically analyzing tutor responses across configurable pedagogical dimensions.The architecture comprises three core components:(1) a rubric-guided large language model (LLM) agent that performs lexical and dialogic cue extraction in a self-reflective, goal-driven manner; (2) a cue-complexity assessment and routing mechanism that sends high-confidence cases to a fine-tuned T5 classifier and escalates low-confidence or ambiguous cases to a reasoning-intensive LLM judge; and (3) an LLM-as-a-judge module that performs structured, multi-step reasoning: (i) generating a domain-grounded reference solution, (ii) identifying conceptual, procedural and cognitive gaps in student output, (iii) inferring the tutor's instructional intent, and (iv) applying the rubric to produce justification-backed classifications.Results show that this unique combination of LLM powered feature engineering, strategic routing and rubrics for grading, enables competitive performance without sacrificing interpretability and cost effectiveness.
Authors
- Ramesh Jain (ORCID: https://orcid.org/0000-0003-2373-4966)
- Srinivasan Rengarajan
Publication Details
- Published
- 2025-01-01
- DOI
- https://doi.org/10.18653/v1/2025.bea-1.86
- Primary Topic
- Intelligent Tutoring Systems and Adaptive Learning
- Type
- article
- Field-Weighted Citation Impact
- 0.00