The ChatGPT Fact-Check: Exploiting the Limitations of Generative AI to Develop Evidenced-Based Reasoning Skills in College Science Courses

Generative artificial intelligence (AI) encompasses a broad spectrum of technologies that create content of various formats. Specifically, large language models (LLMs), which include generative pre-trained transformers (GPT) such as ChatGPT by OpenAI, have revolutionized text-based tasks. Instructors are now challenged with whether to integrate the technology in the classroom or ignore it altogether. Our objective was to determine if LLMs can be used to develop evidence-based reasoning skills in college science courses. LLMs generate fast and informative responses. Of concern, however, is the accuracy of information provided in the responses to a generative AI prompt. LLMs are built on deep learning models that generate text based on language patterns learned in their pre-training datasets. Therefore, information in their responses, as well as sources in the response, can often be inaccurate or even false, termed “hallucination”. We hypothesized that the tendency for AI to generate inaccurate information could be leveraged to develop students’ abilities to critically evaluate claims based on evidence. We thus developed an adaptable assignment called the “ChatGPT Fact-Check” to teach students in college science courses the benefits of using LLMs for topic exploration while exposing them to the inherent limitations of the technology. The assignment requires students to use ChatGPT to generate essays, evaluate AI-generated sources, and assess the validity of AI-generated scientific claims. Here, we present an example of the activity in the context of an upper division Cellular and Molecular Physiology class where students are tasked to explore a monogenic disease of interest, such as cystic fibrosis. ChatGPT was used to generate background essays aimed at exploring the function of the cystic fibrosis transmembrane conductance regulator (CFTR) gene and the molecular and cellular etiology underlying the disease. The LLM was instructed to provide in-text citations for its statements, from which students would evaluate the validity of each source. Students are then tasked with focusing on a specific claim and primary research article generated and cited by ChatGPT, respectively. After evaluating the experimental data within the primary research article, students must apply their evidence-based reasoning skills to determine if the statement made by ChatGPT is supported, partially supported, or refuted by the data provided. In this cystic fibrosis example, we identified errors in the citations generated by ChatGPT and also determined that the evidence cited by the LLM did not actually support the associated claim. In conclusion, we created an assignment to help introduce students to the benefits of AI for general topic exploration, while developing their abilities to distinguish evidence types and analyze a claim based on experimental data. Its adaptable nature allows integration across diverse courses to teach students to responsibly use AI technology for learning while maintaining scientific skepticism. This abstract was presented at the American Physiology Summit 2025 and is only available in HTML format. There is no downloadable file or PDF version. The Physiology editorial board was not involved in the peer review process.