A Comparative Study of the Accuracy and Error/Hallucination Types of Generative AI Chatbot Responses to Local Tourism Information Questions: Focusing on Selected Tourist Attractions in Wonju, South Korea

This exploratory study evaluates the factual accuracy and error/hallucination types of three commercial generative AI chatbots-ChatGPT, Gemini, and Perplexity-when answering local tourism information questions about purposively selected attractions in Wonju, South Korea.Wonju was chosen as a local-city case because heterogeneous tourism assets coexist with facility and access information that has recently changed.Researchers developed eighteen standardized questions for six attractions, covering location/accessibility, operating conditions, and historical or unique attributes.Official materials issued by local government, public agencies, and facility operators were used to construct the reference answers.Fifty-four first responses were requested on July 26, 2026, using new sessions and a common prompt.The responses were segmented into independently verifiable factual claims and judged against the reference evidence.One Gemini response was lost due to user-interface collection artefact; because no response text remained, it was treated as missing data rather than as a chatbot error.Of 92 evaluable factual claims, 89 were accurate, yielding a sample accuracy rate of 96.7%.No hallucination was observed in this limited dataset.The substantive errors comprised one recency error related to recently changed location information and two contradictory claims involving the interpretation of historical dates.At the response level, no statistically significant service differences among the 17 complete matched items.These findings are restricted to the sampled attractions, questions, services, and collection date.The study demonstrates that domain-specific evaluations of generative AI tourism information should distinguish factual accuracy, timeliness, source-context interpretation, and collection integrity, while avoiding broad generalization from a small exploratory sample.

Authors

Institutions

Publication Details

Journal
Asia-pacific Journal of Convergent Research Interchange
Published
2026-10-29
DOI
https://doi.org/10.47116/apjcri.2026.10.12
Primary Topic
AI in Service Interactions
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

A Comparative Study of the Accuracy and Error/Hallucination Types of Generative AI Chatbot Responses to Local Tourism Information Questions: Focusing on Selected Tourist Attractions in Wonju, South Korea

Dong Jin Ku
1 links
Asia-pacific Journal of Convergent Research Interchange
AI in Service Interactions
article

A Comparative Study of the Accuracy and Error/Hallucination Types of Generative AI Chatbot Responses to Local Tourism Information Questions: Focusing on Selected Tourist Attractions in Wonju, South Korea

Dong Jin Ku
article en

Abstract

This exploratory study evaluates the factual accuracy and error/hallucination types of three commercial generative AI chatbots-ChatGPT, Gemini, and Perplexity-when answering local tourism information questions about purposively selected attractions in Wonju, South Korea.Wonju was chosen as a local-city case because heterogeneous tourism assets coexist with facility and access information that has recently changed.Researchers developed eighteen standardized questions for six attractions, covering location/accessibility, operating conditions, and historical or unique attributes.Official materials issued by local government, public agencies, and facility operators were used to construct the reference answers.Fifty-four first responses were requested on July 26, 2026, using new sessions and a common prompt.The responses were segmented into independently verifiable factual claims and judged against the reference evidence.One Gemini response was lost due to user-interface collection artefact; because no response text remained, it was treated as missing data rather than as a chatbot error.Of 92 evaluable factual claims, 89 were accurate, yielding a sample accuracy rate of 96.7%.No hallucination was observed in this limited dataset.The substantive errors comprised one recency error related to recently changed location information and two contradictory claims involving the interpretation of historical dates.At the response level, no statistically significant service differences among the 17 complete matched items.These findings are restricted to the sampled attractions, questions, services, and collection date.The study demonstrates that domain-specific evaluations of generative AI tourism information should distinguish factual accuracy, timeliness, source-context interpretation, and collection integrity, while avoiding broad generalization from a small exploratory sample.

Asia-pacific Journal of Convergent Research InterchangeVol. 12(10)
Sangji University (KR)
Openalex Percentile: Top 14%
AI in Service Interactions
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon