A Comparative Study of the Accuracy and Error/Hallucination Types of Generative AI Chatbot Responses to Local Tourism Information Questions: Focusing on Selected Tourist Attractions in Wonju, South Korea
This exploratory study evaluates the factual accuracy and error/hallucination types of three commercial generative AI chatbots-ChatGPT, Gemini, and Perplexity-when answering local tourism information questions about purposively selected attractions in Wonju, South Korea.Wonju was chosen as a local-city case because heterogeneous tourism assets coexist with facility and access information that has recently changed.Researchers developed eighteen standardized questions for six attractions, covering location/accessibility, operating conditions, and historical or unique attributes.Official materials issued by local government, public agencies, and facility operators were used to construct the reference answers.Fifty-four first responses were requested on July 26, 2026, using new sessions and a common prompt.The responses were segmented into independently verifiable factual claims and judged against the reference evidence.One Gemini response was lost due to user-interface collection artefact; because no response text remained, it was treated as missing data rather than as a chatbot error.Of 92 evaluable factual claims, 89 were accurate, yielding a sample accuracy rate of 96.7%.No hallucination was observed in this limited dataset.The substantive errors comprised one recency error related to recently changed location information and two contradictory claims involving the interpretation of historical dates.At the response level, no statistically significant service differences among the 17 complete matched items.These findings are restricted to the sampled attractions, questions, services, and collection date.The study demonstrates that domain-specific evaluations of generative AI tourism information should distinguish factual accuracy, timeliness, source-context interpretation, and collection integrity, while avoiding broad generalization from a small exploratory sample.
Authors
- Dong Jin Ku (ORCID: https://orcid.org/0000-0002-1076-7403)
Institutions
- Sangji University (KR)
Publication Details
- Journal
- Asia-pacific Journal of Convergent Research Interchange
- Published
- 2026-10-29
- DOI
- https://doi.org/10.47116/apjcri.2026.10.12
- Primary Topic
- AI in Service Interactions
- Type
- article
- Field-Weighted Citation Impact
- 0.00