A framework to synthetically generate fine-grained hallucinated data

The emergence of large language models (LLMs) has transformed natural language processing (NLP), enabling significant advances in text generation and comprehension. However, LLMs often produce hallucinated content-text that is unverifiable, incorrect, or inconsistent with the input-which is especially problematic in high-stakes domains like medicine, law, and finance. While prior research has examined hallucinations in isolated tasks or models, a systematic, fine-grained understanding of hallucination types across critical domains remains lacking. In this work, we develop a methodology to synthetically generate hallucinated question-answer instances categorized into five distinct types: Factual Inconsistency, Factual Fabrication, Logical Inconsistency, Instruction Inconsistency, and Context Inconsistency. To enable this, we construct a benchmark dataset comprising 52,646 hallucinated instances from the medical, legal, and finance domains, using domain-specific datasets (MedQuAD, LegalQAv1, FIQA) for four hallucination types, and datasets with explicit context (CUAD-QA, PubMedQA) for context inconsistency. We evaluate multiple open- and closed-source LLMs on hallucination detection, finding that off-the-shelf models struggle to accurately detect hallucinated content. Fine-tuning LLMs on our dataset leads to significant performance improvements, boosting detection accuracy by 15.8%, 11.4%, and 11.6% in the medical, legal, and finance domains respectively. Our results highlight the importance of structured hallucination training data for building more reliable LLM-based systems.