Chatbots in education: Outperforming students but perceived as less trustworthy

• Care plans labeled as chatbot-generated experience decreased trust. • ChatGPT outperforms students alone and students using ChatGPT in care plan creation. • Discrepancy between perceived trustworthiness and actual chatbot performance. In recent years, chatbots have emerged as potential individual assistance systems in educational settings. However, their successful implementation depends on their ability to deliver high-quality outcomes and gain students’ trust. Against this background, this study investigated the perceived trustworthiness of chatbots in nursing education and whether this trust was justified in light of chatbots’ actual performance in care plan creation. We conducted a study with 189 vocational nursing students who created medical care plans under two conditions: without ChatGPT (GPT-4) and with ChatGPT assistance. Additionally, ChatGPT was used to develop care plans without student involvement. Expert evaluations of these care plans allowed us to compare quality across conditions. We also examined students’ trust by having them evaluate another care plan before and after experimentally manipulating information about its source (chatbot or peer-generated). Statistical analyses revealed that source disclosure significantly affected students’ trust: Care plans believed to be chatbot-generated experienced a significant decrease in trust, while those attributed to peers showed no significant change. However, analyses of expert evaluations showed that ChatGPT-generated plans were of higher quality than those created by students, even when students used ChatGPT assistance. This discrepancy between perceived trustworthiness and actual performance of chatbots indicates that students’ skepticism toward chatbot-generated content is, to some extent, exaggerated. While ChatGPT can enhance students’ care plan quality, our findings emphasize the importance of addressing trust issues in educational settings where chatbots are implemented.