Evaluating human-like chatbot interaction in customer service: integrating anthropomorphism and conversational capability

The development of artificial intelligence (AI)-based technology has driven the development of rule-based chatbots into AI-based chatbots. This development is used by various companies to use AI chatbots as the frontline of customer service. Meeting user expectations for AI chatbots requires an evaluation model to measure chatbot quality. Based on the literature, chatbot evaluation models still focus solely on technical quality, even as AI chatbots are evolving towards human-like services. This research will explore the integration of the technical dimensions of information system success with a new dimension, namely Human-Like Chatbot Interaction (HLCI) as a higher-order construct of anthropomorphism and conversational capability. This quantitative study uses an explanatory research approach. Data were collected through a cross-sectional survey of users of the Veronika chatbot, owned by Telkomsel, an Indonesian telecommunications company. A total of 213 respondents had data ready for analysis using the Partial Least Squares Structural Equation Modeling (PLS-SEM) method in SmartPLS 4, namely the measurement model and structural model tests. The analysis results show that service quality is not significantly related to user satisfaction but has a direct effect on trust. Meanwhile, information quality, system quality, and human-like chatbot interaction are positively related to trust and user satisfaction. Trust and user satisfaction are positively related to intention to continue using. The results of this study can contribute both theoretically and practically to modern AI chatbot quality evaluation models.