HalluSense: A Span-Level Hallucination Detection

Many question-answering systems first look up passages in a document collection and then write their reply fromwhat they found. This design is known as Retrieval-AugmentedGeneration (RAG), and it is meant to keep answers tied to realsources. Even so, such systems still produce smooth sentencesthat the retrieved text does not support. Existing detectorsusually return one verdict for the entire answer, and they saynothing about the reason for the failure, which may lie in thesearch step, in the language model, or in both. We presentHalluSense, an auditing layer placed on top of a RAG pipeline.It cuts the answer into atomic claims, checks every claim againstthe retrieved text with an NLI cross-encoder, calibrates thescores with isotonic regression, and labels each flagged claimas retrieval-caused, generation-misstatement or ungrounded. AStreamlit dashboard shows the evidence, a retrieval-rank trace,a fix hint and a trust score, while a FastAPI service keeps everyaudit in SQLite. On a held-out RAGTruth sample the calibrateddetector reaches a sentence-level AUROC of 0.816 and lowers theexpected calibration error from 0.954 to 0.045. On a 150-caseattribution benchmark it reaches 83.3% three-class and 84.0%two-class accuracy. A full audit takes about 16 seconds on alaptop, and nearly all of that time goes to the local languagemodel.

Authors

Institutions

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-11-18
DOI
https://doi.org/10.5281/zenodo.24349104
Primary Topic
Topic Modeling
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

HalluSense: A Span-Level Hallucination Detection

Yash Yash Sureja, Manthan Manthan Sasane, Shravan Shravan Mehta, Rohan Rohan Divekar et al.
1 links
Zenodo (CERN European Organization for Nuclear Research)
Topic Modeling
preprint

HalluSense: A Span-Level Hallucination Detection

Yash Yash Sureja, Manthan Manthan Sasane, Shravan Shravan Mehta, Rohan Rohan Divekar, Rupali Bhatkhande, Rupali Ma'am
preprint en

Abstract

Many question-answering systems first look up passages in a document collection and then write their reply fromwhat they found. This design is known as Retrieval-AugmentedGeneration (RAG), and it is meant to keep answers tied to realsources. Even so, such systems still produce smooth sentencesthat the retrieved text does not support. Existing detectorsusually return one verdict for the entire answer, and they saynothing about the reason for the failure, which may lie in thesearch step, in the language model, or in both. We presentHalluSense, an auditing layer placed on top of a RAG pipeline.It cuts the answer into atomic claims, checks every claim againstthe retrieved text with an NLI cross-encoder, calibrates thescores with isotonic regression, and labels each flagged claimas retrieval-caused, generation-misstatement or ungrounded. AStreamlit dashboard shows the evidence, a retrieval-rank trace,a fix hint and a trust score, while a FastAPI service keeps everyaudit in SQLite. On a held-out RAGTruth sample the calibrateddetector reaches a sentence-level AUROC of 0.816 and lowers theexpected calibration error from 0.954 to 0.045. On a 150-caseattribution benchmark it reaches 83.3% three-class and 84.0%two-class accuracy. A full audit takes about 16 seconds on alaptop, and nearly all of that time goes to the local languagemodel.

Zenodo (CERN European Organization for Nuclear Research)
MIT Art, Design and Technology University (IN)
Topic Modeling
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon