SYSUpporter Team at BEA 2025 Shared Task: Class Compensation and Assignment Optimization for LLM-generated Tutor Identification

In this paper, we propose a novel framework for the tutor identification track of the BEA 2025 shared task (Track 5).Our framework integrates data-algorithm co-design, dynamic class compensation, and structured prediction optimization.Specifically, our approach employs noise augmentation, a fine-tuned DeBERTa-v3small model with inverse-frequency weighted loss, and Hungarian algorithm-based label assignment to address key challenges, such as severe class imbalance and variable-length dialogue complexity.Our method achieved 0.969 Macro-F1 score on the official test set, securing second place in this competition.Ablation studies revealed significant improvements: a 9.4% gain in robustness from data augmentation, a 5.3% boost in minority-class recall thanks to the weighted loss, and a 2.1% increase in Macro-F1 score through Hungarian optimization.This work advances the field of educational AI by providing a solution for tutor identification, with implications for quality control in LLM-assisted learning environments.

Authors

Publication Details

Published
2025-01-01
DOI
https://doi.org/10.18653/v1/2025.bea-1.83
Primary Topic
Intelligent Tutoring Systems and Adaptive Learning
Type
article
Field-Weighted Citation Impact
0.00

Funders

Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

SYSUpporter Team at BEA 2025 Shared Task: Class Compensation and Assignment Optimization for LLM-generated Tutor Identification

Zeyu Huang, Zheng Xiao, Longfeng Chen, Jin Xu et al.
Intelligent Tutoring Systems and Adaptive Learning
article

SYSUpporter Team at BEA 2025 Shared Task: Class Compensation and Assignment Optimization for LLM-generated Tutor Identification

Zeyu Huang, Zheng Xiao, Longfeng Chen, Jin Xu, Zeng YaWen
article en

Abstract

In this paper, we propose a novel framework for the tutor identification track of the BEA 2025 shared task (Track 5).Our framework integrates data-algorithm co-design, dynamic class compensation, and structured prediction optimization.Specifically, our approach employs noise augmentation, a fine-tuned DeBERTa-v3small model with inverse-frequency weighted loss, and Hungarian algorithm-based label assignment to address key challenges, such as severe class imbalance and variable-length dialogue complexity.Our method achieved 0.969 Macro-F1 score on the official test set, securing second place in this competition.Ablation studies revealed significant improvements: a 9.4% gain in robustness from data augmentation, a 5.3% boost in minority-class recall thanks to the weighted loss, and a 2.1% increase in Macro-F1 score through Hungarian optimization.This work advances the field of educational AI by providing a solution for tutor identification, with implications for quality control in LLM-assisted learning environments.

National Natural Science Foundation of China, National Key Research and Development Program of China
Openalex Percentile: Top 90%
Intelligent Tutoring Systems and Adaptive Learning
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.