Investigating the Impact of a Collaborative Conversational Agent on Dialogue Productivity and Knowledge Acquisition

Abstract While teachers can potentially guide single student-student dialogue, scaling personalized guidance across multiple groups is challenging. Collaborative Conversational Agents (CCAs) emerge as a viable solution, for example, by utilizing learning analytics to identify patterns in student dialogue to trigger interventions grounded in dialogic instructional theory. This study investigates the impact of a CCA, named “Clair,” in a collaborative inquiry learning environment. Building on prior work with Clair, we explored its impact on dialogue productivity and knowledge acquisition. Student pairs were assigned to interact either with Clair (treatment) or without Clair (control), while working in dyads in an inquiry-based digital learning environment on the digestive system. In the analysis of dialogue productivity, a sequential pattern mining technique was employed to measure the frequency of key goals targeted by Clair. Knowledge acquisition was measured through post-tests. Our findings revealed that Clair promoted key goals of dialogue productivity, including: sharing thoughts; orienting and listening to one another; and engaging with each other’s reasoning, but did not have an impact on deepening reasoning. No effect on knowledge acquisition was found. Correlation analysis showed that engaging with each other’s reasoning was related to students’ post‑test scores, but exclusively in the control condition. This may imply that dialogue productivity gains from interacting with Clair did not directly transfer to knowledge acquisition as measured in the post‑test. Following these outcomes, we indicate possible paths for future work to contribute towards a more comprehensive impact of CCAs on dialogue productivity and knowledge acquisition in collaborative learning environments.