HeteroQA-IB: Effective Question Answering on Heterogeneous Data Sources via Information Bottleneck Filtering

This paper presents HeteroQA-IB, a novel framework for hybrid question answering (HQA) that integrates information bottleneck (IB) theory with dynamic program-based prompting to efficiently manage and reason over heterogeneous data sources. By leveraging IB theory, the proposed method enhances the model's ability to focus on relevant information by maximizing the mutual information between the compressed representation and the desired output, while simultaneously minimizing irrelevant details from the input data. HeteroQA-IB is designed to handle diverse data types, including text, images, and tables, and adapt to the complexity of each query. Extensive experiments on multiple benchmark datasets, including Natural Questions (NQ), TRIVIAQA, and HOTPOTQA, show that HeteroQA-IB significantly improves both the accuracy and conciseness of generated answers, outperforming traditional retrieval-augmented models and static prompt-based systems.