Evaluating Pretrained Embeddings for Automatic Short Answer Grading

Automatic Short Answer Grading (ASAG) refers to the computer methods employed to assess student responses based on a given question and the expected answer. Prior studies employed various techniques such as idea mapping, facet mapping, and standard word embeddings to extract semantic information. The researchers manually retrieved different features in order to train the models using the respective datasets. In this study, we evaluate the effectiveness of pretrained embeddings using transfer learning models, including Word2Vec, USE, BERT, and GloVe for the purpose of assessing their performance for ASAG. Our training process involves utilizing cosine similarity derived from the embeddings of the aforementioned models. The error scores and correlation values of the four models are compared with those of prior studies on the Mohler dataset. The findings of our study indicate that the USE model exhibited superior performance compared to the other three models. An examination of potential factors contributing to suboptimal performance of such models is analyzed.