Improving Embeddings Representation via QED Approach for Enhancing Graduate on Time and Employability Prediction

Abstract Embeddings have been widely implemented with Large Language Models in various domains to enhance Machine Learning models prediction compared to existing encoding methods. Due to the high-dimensional embeddings generated, researchers have employed data dimensionality reduction algorithms to improve the embeddings representation, but it remains challenging in interpreting the original features. The difficulties become more significant in education, especially in predicting graduate on time and employability. Therefore, this work proposed Quantized Embeddings Reduction to improve the embeddings representation in predicting graduate on time and employability. This algorithm is compared against existing data dimensionality reduction algorithms under three scenarios based on 4007 graduates across 38 variables. Without feature selection, Quantized Embeddings Reduction achieved the highest accuracy in 9 of 12 semesters for on-time graduation prediction and 10 of 12 semesters for employability prediction. With feature selection, it retained dominance, leading accuracy in 8 sesmesters for both tasks which was peaking at 81.3% in employability prediction while achieving unmatched precision with 88.2% compared to others with less than 88.1% in on-time graduation and F1-score with 89.5% compared to others with less than 89.4%. Under feature selection and hyperparameter tuning, the algorithm secured top accuracy in 11 semesters in employability prediction while dominating precision (88.1%). A Friedman test ranked the proposed algorithm as the best-performing algorithm in improving embeddings representation. In future, the scalability of proposed algorithm could be extended to broader domains.