Explainable Hallucination through Natural Language Inference Mapping
Large language models (LLMs) often generate hallucinated content, making it crucial to identify and quantify inconsistencies in their outputs.We introduce HaluMap, a post-hoc framework that detects hallucinations by mapping entailment and contradiction relations between source inputs and generated outputs using a natural language inference (NLI) model.To improve reliability, we propose a calibration step leveraging intra-text relations to refine predictions.HaluMap outperforms stateof-the-art NLI-based methods by five percentage points compared to other training-free approaches, while providing clear, interpretable explanations.As a training-free and modelagnostic approach, HaluMap offers a practical solution for verifying LLM outputs across diverse NLP tasks.The resources of this paper are available at https://github.com/ caisa-lab/acl25-halumap.
Authors
- Akbar Karimi (ORCID: https://orcid.org/0000-0002-5132-2435)
- Lucie Flek (ORCID: https://orcid.org/0000-0002-5995-8454)
- Zhixue Zhao (ORCID: https://orcid.org/0000-0002-3060-269X)
- Wei-Fan Chen (ORCID: https://orcid.org/0000-0003-3400-6075)
Publication Details
- Published
- 2025-01-01
- DOI
- https://doi.org/10.18653/v1/2025.findings-acl.96
- Primary Topic
- Mental Health Research Topics
- Type
- article
- Field-Weighted Citation Impact
- 0.00
Funders
- Bundesministerium für Bildung und Forschung