Exploring students’ (mis)conceptions about ChatGPT-generated text: A qualitative study

With the advent of large language models (LLMs), a major concern has become how students can be taught artificial intelligence (AI) literacy. Arguably, a necessary starting point for teaching AI literacy is knowledge about students’ pre-existing understanding of how LLMs work and the output they generate. Therefore, the main purpose of this study was to explore students’ understanding of ChatGPT-generated text, that is, their understanding of the origin of ChatGPT’s information and of how ChatGPT uses that information in generating textual output. Two hundred sixty-seven Norwegian high-school students responded in writing to broad open-ended questions about these issues, and we used thematic analysis in qualitatively exploring their conceptions and misconceptions. Students held a range of adequate and adaptive conceptions regarding the sources, the output, the function, and the use of ChatGPT-generated text. However, they also held several misconceptions that may hamper constructive use of ChatGPT as a literacy and learning tool. In particular, there was considerable confusion regarding the information sources that ChatGPT draws on in generating text, with an overestimation of the general quality of those sources also associated with a tendency to put unwarranted trust in ChatGPT-generated textual output. Potential implications concern both the conceptualization of AI literacy and efforts to promote such literacy among students.