Explainable Hallucination through Natural Language Inference Mapping

Large language models (LLMs) often generate hallucinated content, making it crucial to identify and quantify inconsistencies in their outputs.We introduce HaluMap, a post-hoc framework that detects hallucinations by mapping entailment and contradiction relations between source inputs and generated outputs using a natural language inference (NLI) model.To improve reliability, we propose a calibration step leveraging intra-text relations to refine predictions.HaluMap outperforms stateof-the-art NLI-based methods by five percentage points compared to other training-free approaches, while providing clear, interpretable explanations.As a training-free and modelagnostic approach, HaluMap offers a practical solution for verifying LLM outputs across diverse NLP tasks.The resources of this paper are available at https://github.com/ caisa-lab/acl25-halumap.

Authors

Publication Details

Published
2025-01-01
DOI
https://doi.org/10.18653/v1/2025.findings-acl.96
Primary Topic
Mental Health Research Topics
Type
article
Field-Weighted Citation Impact
0.00

Funders

Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

Explainable Hallucination through Natural Language Inference Mapping

Akbar Karimi, Lucie Flek, Zhixue Zhao, Wei-Fan Chen
Mental Health Research Topics
article

Explainable Hallucination through Natural Language Inference Mapping

Akbar Karimi, Lucie Flek, Zhixue Zhao, Wei-Fan Chen
article en

Abstract

Large language models (LLMs) often generate hallucinated content, making it crucial to identify and quantify inconsistencies in their outputs.We introduce HaluMap, a post-hoc framework that detects hallucinations by mapping entailment and contradiction relations between source inputs and generated outputs using a natural language inference (NLI) model.To improve reliability, we propose a calibration step leveraging intra-text relations to refine predictions.HaluMap outperforms stateof-the-art NLI-based methods by five percentage points compared to other training-free approaches, while providing clear, interpretable explanations.As a training-free and modelagnostic approach, HaluMap offers a practical solution for verifying LLM outputs across diverse NLP tasks.The resources of this paper are available at https://github.com/ caisa-lab/acl25-halumap.

Bundesministerium für Bildung und Forschung
Openalex Percentile: Top 80%
Mental Health Research Topics
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.