Embracing LLM Feedback: the role of feedback providers and provider information for feedback effectiveness

Feedback is an integral part of learning in higher education and is increasingly being provided to students via modern technologies like Large Language Models (LLMs). But students’ perception of feedback from LLMs vs. feedback from educators remains unclear even though it is an important facet of feedback effectiveness. Further, feedback effectiveness can be negatively influenced by various factors; For example, (not) knowing certain characteristics about the feedback provider may bias a student’s reaction to the feedback process. To assess perceptions of LLM feedback and mitigate the negative effects of possible biases, this study investigated the potential of providing provider-information about feedback providers. In a 2×2 between-subjects design with the factors feedback provider (LLM vs. educator) and provider-information (yes vs. no), 169 German students evaluated feedback message and provider perceptions. Path analyses showed that the LLM was perceived as more trustworthy than an educator and that the provision of provider-information led to improved perceptions of the feedback. Furthermore, the effect of the provider and the feedback on perceived trustworthiness and fairness changed when provider-information was provided. Overall, our study highlights the importance of further research on feedback processes that include LLMs due to their influential nature and suggests practical recommendations for designing digital feedback processes.