Towards Trustworthy Conversational Agents: A New Method for Hallucination Detection in NLG

Conversational agents, powered by Large Language Models (LLMS) have seen significant advancements, enabling them to produce coherent and contextually relevant text. However, a major challenge remains in the form of Hallucination, where the LLMs generate content that is fabricated and factually incorrect. This issue is especially critical in domains that require high factual accuracy, where it is essential to ensure the reliability and trustworthiness of LLM-generated content. This paper provides a novel metric based method towards detection of hallucinations in natural language generation. The method combines three complementary components: Jaccard similarity for entity-level alignment, natural language inference for semantic consistency, and a normalized perplexity measure for fluency. The integration of these components through a weighted average produces a hallucination metric where a high value indicates a potential occurrence of hallucination. This metric increases the reliability of LLM-generated content. The method’s effectiveness is evaluated on benchmark datasets (BEGIN and FEVER), which demonstrates improved performance in hallucination detection. The evaluation results indicate that the proposed method provides a balanced approach to identifying hallucinations, offering a promising solution for more trustworthy conversational agents.