Evaluating Relation Hallucination in Text Summarization: An Introduction to the Relation Hallucination Index

Text summarization involves generating a concise, precise, and coherent summary that captures the essence of a longer document.Relation Hallucination refers to the generation of summary sentences that imply or state relationships between entities or events that were not present or implied in the original document.Relation Hallucination is a pivotal concern in abstractive text summarization, where models fabricate or exaggerate connections, leading to misleading summaries.This paper delves deep into the evaluation of Relation Hallucination in abstractive summarization.We introduce a novel metric, the Relation Hallucination Index , designed to evaluate and quantify Relation hallucinations across various state-of-the-art models.Emphasizing the paramount importance of maintaining accurate relational context in summaries, this article showcases the efficacy of the Relation hallucination index in discerning fabricated relationships.The Relation hallucination index provides a quantitative measure of the various levels of Relation Hallucination present in generated summaries, enabling practitioners from industry as well as Academia to select models aligned with desired hallucination parameters for their tailored applications. CCS Concepts General and reference