Faithfulness- and Distribution-Aware Text Analytics for Business Intelligence: From Evidence Retrieval to Plain-Language Adaptation

Business Intelligence systems increasingly rely on complex unstructured evidence, creating a need to retrieve relevant information while presenting it in an accessible form without obscuring decision-relevant content. This study presents a faithfulness- and distribution-aware evidence-to-insight framework that integrates sparse and dense retrieval, neural reranking, plain-language adaptation, reference-free candidate selection, interpretable source-consistency screening, provenance, and conservative source fallback. The empirical evaluation is deliberately component-wise: SciFact is used to evaluate evidence retrieval, PLABA provides an untouched in-domain evaluation of document-level plain-language adaptation and reliability-aware selection, and SimpleText2024 provides a frozen external-transfer evaluation under dataset and input-structure differences. The framework is termed faithfulness-aware in a restricted operational sense because it screens numerical, negation, and uncertainty consistency and can abstain when predefined requirements are not satisfied; it does not provide comprehensive factuality verification. On SciFact, cross-encoder reranking achieves an MRR of 0.8536, compared with 0.7675 for BM25. On the untouched PLABA test partition, Beam-4 achieves a grouped SARI of 46.595, compared with 46.137 for the frozen reference-free selector. Under frozen transfer to SimpleText2024, Greedy becomes the strongest fixed decoder, achieving a sentence-level SARI of 21.046. These findings show that development-time adaptation and selection preferences should not be assumed to remain stable on unseen or structurally different data. The resulting framework therefore distinguishes a simpler quality-first operating mode from a more conservative reliability-first mode that combines consistency screening, traceable provenance, and source fallback.

Authors

Institutions

Publication Details

Journal
Information
Published
2026-11-18
DOI
https://doi.org/10.3390/info17111169
Primary Topic
Topic Modeling
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Faithfulness- and Distribution-Aware Text Analytics for Business Intelligence: From Evidence Retrieval to Plain-Language Adaptation

Avi Arampatzis, Georgios Arampatzis
Information
Topic Modeling
article

Faithfulness- and Distribution-Aware Text Analytics for Business Intelligence: From Evidence Retrieval to Plain-Language Adaptation

Avi Arampatzis, Georgios Arampatzis
article en

Abstract

Business Intelligence systems increasingly rely on complex unstructured evidence, creating a need to retrieve relevant information while presenting it in an accessible form without obscuring decision-relevant content. This study presents a faithfulness- and distribution-aware evidence-to-insight framework that integrates sparse and dense retrieval, neural reranking, plain-language adaptation, reference-free candidate selection, interpretable source-consistency screening, provenance, and conservative source fallback. The empirical evaluation is deliberately component-wise: SciFact is used to evaluate evidence retrieval, PLABA provides an untouched in-domain evaluation of document-level plain-language adaptation and reliability-aware selection, and SimpleText2024 provides a frozen external-transfer evaluation under dataset and input-structure differences. The framework is termed faithfulness-aware in a restricted operational sense because it screens numerical, negation, and uncertainty consistency and can abstain when predefined requirements are not satisfied; it does not provide comprehensive factuality verification. On SciFact, cross-encoder reranking achieves an MRR of 0.8536, compared with 0.7675 for BM25. On the untouched PLABA test partition, Beam-4 achieves a grouped SARI of 46.595, compared with 46.137 for the frozen reference-free selector. Under frozen transfer to SimpleText2024, Greedy becomes the strongest fixed decoder, achieving a sentence-level SARI of 21.046. These findings show that development-time adaptation and selection preferences should not be assumed to remain stable on unseen or structurally different data. The resulting framework therefore distinguishes a simpler quality-first operating mode from a more conservative reliability-first mode that combines consistency screening, traceable provenance, and source fallback.

InformationVol. 17(11)
Democritus University of Thrace (GR)
Topic Modeling
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon