Developing a generative AI chatbot to support students’ higher-order thinking in STEM education

ABSTRACT Higher-order thinking is a key component of science, technology, engineering, mathematics (STEM) reasoning and supports students’ ability to reflect on their ideas, make connections, and generalize across contexts. We report a new approach to the use of generative AI in STEM education, in which we are developing a chatbot that aims to engage students in higher-order thinking, rather than giving instructional content. We describe a proof of concept test of whether interaction with our generative artificial intelligence chatbot could elicit students’ higher-order thinking talk (HOTT) in conversation as they reflected on their solutions to proportional reasoning problems. Thirty-three fifth- and sixth-grade students received a 30-min lesson on symbolic and number line strategies for comparing ratios and completed an assessment following the lesson. Students then engaged in dialog with the UCI Math ThinkBot (afterward Math ThinkBot ) to explain their solutions and compare strategies. These transcripts were examined qualitatively and quantitatively for HOTT, revealing that HOTT appeared in both chatbot prompts and children’s responses—which is important because eliciting student HOTT can be a challenge in classroom discourse. In addition, children’s HOTT was positively associated with the HOTT produced by the chatbot. These findings suggest that human-artificial intelligence dialog can lead to instructional HOTT discourse and may offer a promising approach for supporting higher-order thinking and STEM reasoning.