NLIP at BEA 2025 Shared Task: Evaluation of Pedagogical Ability of AI Tutors
This paper presents our system submission to the Building Educational Applications (BEA) 2025 Shared Task on Pedagogical Ability Assessment of AI-powered Tutors.The task evaluates multiple dimensions of AI tutor responses within student-teacher educational dialogues, including mistake identification, mistake location, providing guidance, and actionability.Our approach leverages transformer-based models (Vaswani et al., 2017), primarily DeBERTa and RoBERTa, and incorporates ordinal regression, threshold tuning, oversampling, and multi-task learning.Our best-performing systems are capable of assessing tutor response quality across all tracks.This highlights the effectiveness of tailored transformer architectures and pedagogically motivated training strategies for AI tutor evaluation.
Authors
- Tapan Kumar Saha (ORCID: https://orcid.org/0000-0003-0763-0032)
- Sagar Ganguli (ORCID: https://orcid.org/0000-0002-1209-1689)
- Maunendra Sankar Desarkar (ORCID: https://orcid.org/0000-0003-1963-7338)
Publication Details
- Published
- 2025-01-01
- DOI
- https://doi.org/10.18653/v1/2025.bea-1.99
- Primary Topic
- Online Learning and Analytics
- Type
- article
- Field-Weighted Citation Impact
- 0.00