Quality counts? Examining the role of feedback provider and feedback quality on students' feedback perceptions

Abstract With the advent of large language models (LLMs) as feedback providers in education, there is a pressing need to investigate the potential effectiveness of the feedback provided by such artificial intelligence (AI) systems. The purpose of this 2 × 2 mixed‐design study was thus to investigate biases that students may have in their perceptions of feedback messages and feedback providers as a function of the feedback provider label (between‐factor; educators vs. LLMs) and the feedback message quality (within‐factor; low [performance evaluation + definition] vs. high [performance evaluation + definition + worked examples + text references]). Data from 173 university students were analysed with linear mixed‐effects models. Results indicated that students processed the predetermined feedback messages in an unbiased manner because they evaluated high‐quality feedback more positively than low‐quality feedback, independently of the feedback provider label. However, the results suggested provider‐induced biases in students' perceptions. For instance, compared to an educator label, an LLM label reduced the perceived trustworthiness of the feedback provider. Nevertheless, and despite the social biases that may occur when students are confronted with feedback that is labelled to originate from an LLM, the quality of the feedback message seems to be the more crucial factor in shaping students' positive perceptions of the feedback messages and the feedback providers.