Impact of criterion-based reflection on prospective physics teachers’ perceptions of ChatGPT-generated content

ChatGPT has significantly shaped digital transformation discussions. Its widespread testing and ongoing optimization highlight the need to assess its capabilities and encourage critical reflection. This study explores how prospective physics teachers’ critical reflection on ChatGPT-generated content impacts their perceptions of its answer quality and further use. Involving 39 prospective physics teachers, the study assessed their evaluations of ChatGPT’s answers to didactical tasks using predefined criteria, referred in this paper as the criterion-based evaluation approach. Pre- and post-questionnaires, with a 5-point ranking scale and open-ended questions, evaluated prospective physics teachers’ perceptions of ChatGPT’s helpfulness and quality. Results showed that critical reflection shifted their perception of answer quality and increased their awareness of ChatGPT’s limitations. Three perspectives on ChatGPT’s further use emerged: extreme positive, extreme negative, and balanced. The extreme positive perspective indicated confirmation bias, where the positive prior experiences with ChatGPT influenced their evaluations. The findings highlight the need to foster critical thinking and media literacy in teacher education to help educators effectively integrate such tools.