Faithfulness- and Distribution-Aware Text Analytics for Business Intelligence: From Evidence Retrieval to Plain-Language Adaptation
Business Intelligence systems increasingly rely on complex unstructured evidence, creating a need to retrieve relevant information while presenting it in an accessible form without obscuring decision-relevant content. This study presents a faithfulness- and distribution-aware evidence-to-insight framework that integrates sparse and dense retrieval, neural reranking, plain-language adaptation, reference-free candidate selection, interpretable source-consistency screening, provenance, and conservative source fallback. The empirical evaluation is deliberately component-wise: SciFact is used to evaluate evidence retrieval, PLABA provides an untouched in-domain evaluation of document-level plain-language adaptation and reliability-aware selection, and SimpleText2024 provides a frozen external-transfer evaluation under dataset and input-structure differences. The framework is termed faithfulness-aware in a restricted operational sense because it screens numerical, negation, and uncertainty consistency and can abstain when predefined requirements are not satisfied; it does not provide comprehensive factuality verification. On SciFact, cross-encoder reranking achieves an MRR of 0.8536, compared with 0.7675 for BM25. On the untouched PLABA test partition, Beam-4 achieves a grouped SARI of 46.595, compared with 46.137 for the frozen reference-free selector. Under frozen transfer to SimpleText2024, Greedy becomes the strongest fixed decoder, achieving a sentence-level SARI of 21.046. These findings show that development-time adaptation and selection preferences should not be assumed to remain stable on unseen or structurally different data. The resulting framework therefore distinguishes a simpler quality-first operating mode from a more conservative reliability-first mode that combines consistency screening, traceable provenance, and source fallback.
Authors
- Avi Arampatzis (ORCID: https://orcid.org/0000-0003-2415-4592)
- Georgios Arampatzis (ORCID: https://orcid.org/0000-0003-0674-6260)
Institutions
- Democritus University of Thrace (GR)
Publication Details
- Journal
- Information
- Published
- 2026-11-18
- DOI
- https://doi.org/10.3390/info17111169
- Primary Topic
- Topic Modeling
- Type
- article
- Field-Weighted Citation Impact
- 0.00