A Comparison of Peer- and Tutor-Grading of an Introductory R Coding Assessment

We investigate the level of agreement between tutor and peer grading of an introductory R programming assessment. Comparing peer and tutor grades we find a strong correlation of 0.848, 95%CI = (0.809, 0.880). Using standard multivariate data analysis techniques we find that tutors and peers grade similarly given a prescriptive criterion. However, when given a subjective criterion, tutors and peers use different schemas to grade. We find that tutors grade the subjective criterion autonomously from the other rubric criteria, whereas peers grade in line with the prescriptive criteria that evaluates the components and structure of the code. In addition, we estimate between-assessor and between-submission variation using a discrete-Beta mixed model and show that between-submission is greater than grader submission for both peers and tutors. Finally, we advocate for the use of peer assessment as a learning exercise and encourage readers to adapt the activity accordingly.