Understanding student perspectives on generative AI chatbots: a human-centred mixed-methods study in higher education

Abstract This study addresses the educational challenges of understanding how generative AI chatbots can equitably and effectively enhance student learning in higher education. Chatbot system analytics data and a mixed methods survey (n = 121) was used to examine four research questions related to the use of chatbots (academic value, frequency of use, and attitudes), technology experience and AI literacy (self-reported digital proficiency, device usage and AI familiarity), barriers to use of chatbots (ethical concerns, UX issues and policy ambiguity), and general user experience (clarity, relevance, accuracy, satisfaction and likelihood of recommendation). Descriptive statistics, one-sample t-tests, and ANOVAs revealed generally positive attitudes toward chatbots and perceived gains in knowledge and understanding, together with robust support for academic-integrity requirements. Usage analytics confirmed 24/7 usage requirements, with 36.8% of interactions occurring after hours. Users reported clear, relevant, largely accurate responses and an easy interface but overall satisfaction was mixed. The study recommends human-centred design (e.g., opt-in launch, clearer links to official resources), explicit AI policies and labels for assessment, staff and student training (including prompt literacy, and potential use cases), subject-specific knowledge bases, and continuous monitoring for errors and edge cases, and subsequent update of system instructions and knowledge bases. Study findings highlight conditions under which chatbots can equitably complement human instruction and methods for chatbot engagement across different student cohorts.