Language Models and Dialect Differences

The advancements in automatic language processing being ushered in by Large Language Models suggest enormous potential for better personalization during student learning. However, this potential can be best exploited if we know that LLMs are equally capable of interacting with students who speak or write in a range of different dialects. This case study uses systematically manipulated student essays, previously evaluated by human raters, to examine how ChatGPT responds to and addresses specific dialect differences. Results point to important concerns about the potential biases and limitations of both LLMs and humans when evaluating and providing feedback to students who use minoritized dialects. Addressing these concerns is critical for the field of learning analytics, as it seeks to ensure equity and asset-based approaches to learning analytics.