Classification of Hallucinations in Large Language Models Using a Novel Weighted Metric

As Large Language Models (LLMs) find increasing use in important fields such as healthcare, finance, and law, ensuring their accuracy and reliability is critical. One significant challenge is the occurrence of “hallucinations,” where these models produce nonsensical or incorrect information. This paper introduces a new framework designed to identify and categorize hallucinations in the outputs of LLMs, particularly in safety-sensitive applications. We present a detailed system that classifies hallucinations into four categories: Factual Errors, Speculative Responses, Logical Fallacies, and Improbable Scenarios. Our methodology employs a scoring system that combines metrics to offer a clearer picture of the model performance. Using the TruthfulQA dataset, and the Falcon 7B model, we analyze different types of hallucinations and their potential to compromise decision making in safety critical domains. By focusing on clarity and accuracy, this framework aims to improve the safety and reliability of LLMs in high stakes situations and sets the stage for more effective validation methods in artificial intelligence.