Let Us Not Squander\nthe Affordances of LLMs for the\nSake of Expedience: Using Retrieval Augmented Generative AI Chatbots\nto Support and Evaluate Student Reasoning

The use of large language model Generative\nAI (GenAI)\nsystems by\nstudents and instructors is increasing rapidly, and there is little\nchoice but to adapt to this new situation. Many, but not all, students\nare using GenAI for homework and assignments, which means that we\nneed to provide equitable access for all students to AI systems that\ncan support and enhance their learning. At the same time, we need\nto think carefully about just what we want teaching and learning to\nlook like as GenAI systems become readily available. Here we propose\nthat “business as usual” is not a responsible option.\nAlthough chatbots can readily answer questions, produce summaries\nof content, and make the process of education more efficient, there\nis scant evidence that such time saving is effective, and indeed,\nit is important that we not allow the use of GenAI systems to circumvent\nor undermine the learning process. The availability of so-called Retrieval\nAugmented Generative (RAG) AI systems allows us to expand what we\nexpect students to know and do, by 1) supporting instructors in the\ndesign of more complex tasks (that can, for example, elicit evidence\nof three-dimensional learning (3DL)), 2) supporting students as they\nreason through such scaffolded tasks, and 3) by evaluating student\nresponses, individually and in aggregate. We present examples of each\nof these affordances with the associated training materials and bot\npersonas, along with caveats about their use.