CU at BEA 2025 Shared Task: A BERT-Based Cross-Attention Approach for Evaluating Pedagogical Responses in Dialogue

Automatic evaluation of AI tutor responses in educational dialogues is a challenging task, requiring accurate identification of mistakes and provision of pedagogically effective guidance.In this paper, we propose a classification model based on BERT, enhanced with a cross-attention mechanism that explicitly models the interaction between the tutor's response and preceding dialogue turns.This design enables better alignment between context and response, supporting more accurate assessment along the educational dimensions defined in the BEA 2025 Shared Task.To address the substantial class imbalance in the dataset, we employ data augmentation techniques for minority classes.Our system consistently outperforms baseline models across all tracks.However, performance on underrepresented labels remains limited, particularly when distinguishing between semantically similar cases.This suggests room for improvement in both model expressiveness and data coverage, motivating future work with stronger decoder-only model and auxiliary information from systems like GPT-4.1.Overall, our findings offer insights into the potential and limitations of LLM-based approaches for pedagogical feedback evaluation.

Authors

Institutions

Publication Details

Published
2025-01-01
DOI
https://doi.org/10.18653/v1/2025.bea-1.81
Primary Topic
Intelligent Tutoring Systems and Adaptive Learning
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

CU at BEA 2025 Shared Task: A BERT-Based Cross-Attention Approach for Evaluating Pedagogical Responses in Dialogue

Zhiyang Lyu
Intelligent Tutoring Systems and Adaptive Learning
article

CU at BEA 2025 Shared Task: A BERT-Based Cross-Attention Approach for Evaluating Pedagogical Responses in Dialogue

Zhiyang Lyu
article en

Abstract

Automatic evaluation of AI tutor responses in educational dialogues is a challenging task, requiring accurate identification of mistakes and provision of pedagogically effective guidance.In this paper, we propose a classification model based on BERT, enhanced with a cross-attention mechanism that explicitly models the interaction between the tutor's response and preceding dialogue turns.This design enables better alignment between context and response, supporting more accurate assessment along the educational dimensions defined in the BEA 2025 Shared Task.To address the substantial class imbalance in the dataset, we employ data augmentation techniques for minority classes.Our system consistently outperforms baseline models across all tracks.However, performance on underrepresented labels remains limited, particularly when distinguishing between semantically similar cases.This suggests room for improvement in both model expressiveness and data coverage, motivating future work with stronger decoder-only model and auxiliary information from systems like GPT-4.1.Overall, our findings offer insights into the potential and limitations of LLM-based approaches for pedagogical feedback evaluation.

University of Colorado Boulder (US), University of Colorado System (US)
Openalex Percentile: Top 90%
Intelligent Tutoring Systems and Adaptive Learning
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

CU at BEA 2025 Shared Task: A BERT-Based Cross-Attention Approach for Evaluating Pedagogical Responses in Dialogue — Zhiyang Lyu · (2025) | TGRS Research Map | TGRS