Advanced Deep Learning Model for Improving Student Evaluation in Programming Education

This paper investigates the potential applications of the pre-trained language model UniXcoder in programming education, with a focus on the utility of its embeddings in evaluating students' solutions. Due to its transformer-based architecture, UniXcoder is capable of generating syntactically and semantically meaningful code embeddings, enabling it to effectively understand and represent code patterns. The study employs these embeddings as input for predictive models to predict the correctness of student submissions. Using the Dublin introductory programming submission dataset (7,490 instances), Singapore dataset (4,393 instances), and Annaba dataset (87 instances), various machine learning models were compared. The primary contribution is the comparison of VniXcoder-based embeddings with traditional feature extraction methods. Results demonstrate that UniXcoder embeddings enhance prediction performance effectively, with the best model being the MLP with UniXcoder embeddings, achieving an accuracy of 96.06%, F1 score of 96.63%, Cohen's Kappa of 0.91, and ROC-AUC of 97.83% for the Dublin dataset. For the Singapore dataset, the MLP model achieved an accuracy of 87.71%, F1 score of 89.09%, Cohen's Kappa of 0.7504, and ROC-AUC of 92.73%. Additionally, the Balanced Bagging model on the Annaba dataset showed strong performance with an accuracy of 94.12%, F1 score of 95.24%, Cohen's Kappa of 0.8759, and ROC-AUC of 98.48%. These findings indicate that UniXcoder embeddings represent a promising approach for improving automated programming assessment.