Design of Automatic Short-Answer Scoring Prediction Model in Academic Courses Based on Statistical and Deep Learning Algorithms

In the digital education system, the increasing demand for measuring lexical and semantic text similarity has led to the need for developing scalable and intelligent automated grading systems. Traditional assessment methods used for short-answer grading are labor-intensive, time-consuming and prone to human-evaluator bias. Furthermore, computer science curricula have different evaluation challenges as programming-centric courses require rigorous syntactical and structural validation, whereas theoretical courses require deep conceptual and semantic comprehension. To address these dual challenges, we introduce the Virtual University Automatic Short-Answer Grading (VUASAG) framework, a model addressing both statistical-based lexical analysis and Transformer-based semantic architectures. Utilizing an expanded Mohler dataset spanning multiple computer science domains (including Data Structures, Introduction to Programming, Object-Oriented Programming, and Software Engineering), we evaluated the efficiency of traditional string-matching metrics alongside state-of-the-art Transformer-based models (T5, BERT, XLNet, and SBERT). The empirical results demonstrate that token-based statistical models perform robustly on keyword-dependent syntax validation while Transformer-based models excel by capturing deep conceptual context. Specifically, Sentence-BERT (SBERT) achieved superior predictive accuracy across courses, yielding a minimum Root Mean Square Error (RMSE) of 0.9511 for the Introduction to Programming course, paired with moderate positive Pearson and Spearman correlation coefficients. Confidence intervals are estimated using Fisher’s z-transformation with a 95 percentile confidence level. Question-level five-fold and two-fold cross-validation is conducted for all four courses, and SBERT is found to perform better. Finally, to demonstrate the practical utility, we deployed a web-based portal for students’ evaluation leveraging a LLaMA-based pre-trained model to deliver real-time, personalized pedagogical feedback measuring code functionality, quality, documentation, and error handling. This reduces the educator workload while preserving grading integrity.