NLIP at BEA 2025 Shared Task: Evaluation of Pedagogical Ability of AI Tutors

This paper presents our system submission to the Building Educational Applications (BEA) 2025 Shared Task on Pedagogical Ability Assessment of AI-powered Tutors.The task evaluates multiple dimensions of AI tutor responses within student-teacher educational dialogues, including mistake identification, mistake location, providing guidance, and actionability.Our approach leverages transformer-based models (Vaswani et al., 2017), primarily DeBERTa and RoBERTa, and incorporates ordinal regression, threshold tuning, oversampling, and multi-task learning.Our best-performing systems are capable of assessing tutor response quality across all tracks.This highlights the effectiveness of tailored transformer architectures and pedagogically motivated training strategies for AI tutor evaluation.

Authors

Publication Details

Published
2025-01-01
DOI
https://doi.org/10.18653/v1/2025.bea-1.99
Primary Topic
Online Learning and Analytics
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

NLIP at BEA 2025 Shared Task: Evaluation of Pedagogical Ability of AI Tutors

Tapan Kumar Saha, Sagar Ganguli, Maunendra Sankar Desarkar
Online Learning and Analytics
article

NLIP at BEA 2025 Shared Task: Evaluation of Pedagogical Ability of AI Tutors

Tapan Kumar Saha, Sagar Ganguli, Maunendra Sankar Desarkar
article en

Abstract

This paper presents our system submission to the Building Educational Applications (BEA) 2025 Shared Task on Pedagogical Ability Assessment of AI-powered Tutors.The task evaluates multiple dimensions of AI tutor responses within student-teacher educational dialogues, including mistake identification, mistake location, providing guidance, and actionability.Our approach leverages transformer-based models (Vaswani et al., 2017), primarily DeBERTa and RoBERTa, and incorporates ordinal regression, threshold tuning, oversampling, and multi-task learning.Our best-performing systems are capable of assessing tutor response quality across all tracks.This highlights the effectiveness of tailored transformer architectures and pedagogically motivated training strategies for AI tutor evaluation.

Quality Education
Openalex Percentile: Top 80%
Online Learning and Analytics
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.