A Study on Methods for Improving the Performance of Descriptive Answer Clustering in Schools: Focusing on Tokenizer and Feature Extraction

A model capable of automatically categorizing or clustering student responses based on solution strategies would significantly benefit the operation of descriptive assessments in schools. Although research on clustering methods for descriptive answers has been conducted, there is still room for improvement as these methods have not fully reflected the unique characteristics of mathematical descriptive answers. Accordingly, this study proposes a method to improve existing clustering approaches by developing a tokenizer more suitable for mathematics and utilizing N-gram extraction of variables in mathematical expressions for clustering. To verify the effectiveness of the proposed method, responses to five descriptive questions were collected, and the performance of clustering models applying the proposed method was compared with existing approaches. The results demonstrated that the new tokenizer and feature extraction method maintained or improved performance across most clustering models. This study is significant in demonstrating that incorporating the characteristics of mathematical descriptive answers into tokenization and variable extraction techniques can enhance clustering performance, providing an effective approach for grading assistance in descriptive assessments.