LLM-based conversational agents for behaviour change support: A randomised controlled trial examining efficacy, safety, and the role of user behaviour

This study examines the use of Motivational Interviewing (MI) principles in a GPT-4-based chatbot, MIcha, to promote behaviour change. We conducted a pre-registered randomised controlled trial to assess the integration of MI techniques in conversational agents, aiming to support users’ behaviour change through guided self-reflection and identify how users interact with large language model (LLM)-based systems in this context. Results indicate that short conversations with LLM-based chatbots are successful at increasing users’ readiness to change and usage of MI principles during text generation can effectively mitigate potential harms. Additionally, we identified distinct user behaviour types — cooperative, reflective, and pre-informed—that significantly influenced the outcomes of interactions. These findings demonstrate the potential of MI principles in enhancing the efficacy of conversational agents for behaviour change and highlight the importance of user behaviour in shaping interaction dynamics. • Brief LLM chats on behaviour change goals significantly boost readiness to change. • Motivational interviewing can significantly reduce potential harms in LLM outputs. • User behaviour types heavily influence conversation outcome and success.