Addressing the Black-Box Challenge: Confidence Index Approach to Grading of Open-Ended Assessments in Accounting Education

Machine learning (ML) application to grading has been gaining traction in educational system given arguments that it has the potential to effectively analyse and evaluate diverse responses typical of open-ended examination, thereby enhancing assessment and feedback processes. By comparing confidence index to experience, the study examined ML grading of open-ended examinations within the accounting education, with an aim to eliminate the black-box effect. Based on a probit modelling, confidence index significantly and positively impacts grading while experience showed mixed effects. Our findings suggest the integration of confidence indices would enhance transparency and interpretability, thereby eliminating black-box effect. The study provides critical steps for transparent ML grading to balance subjective and objective metrics, while ensuring fairness and reliability in academic and professional assessments