Modeling and Analysis of Question Answering for Long Context and Compound Questions Using BERT Model

In the context of Indian Lok Sabha question answering, this study introduces a unique use of the Bidirectional Encoder Representations from Transformers (BERT) paradigm for managing long and compound questions. We introduce a specialized dataset of Lok Sabha questions, addressing the unique challenges of parliamentary language and complex queries. Our approach modifies the BERT architecture with hierarchical attention mechanisms to better process multi-part questions. The model was trained on a carefully curated dataset of Lok Sabha questions spanning 2014-2023, encompassing a wide range of topics relevant to Indian governance. We implemented data augmentation techniques, including question decomposition and combination, to enhance the model’s ability to handle compound queries. Our BERT-based model (BERT-VIndLok) outperformed existing baselines, achieving a 76.3% Exact Match score, 84.2% F1 score, and 72.1% Accuracy on the Lok Sabha dataset. These results demonstrate significant improvements over traditional question answering systems in handling the complexities of parliamentary inquiries. Through detailed error analysis and attention visualizations, we provide insights into the model’s behavior and its effectiveness in understanding question structure. We also identify areas for future research, including improving performance on explanatory questions. This work explores how transformer-based models can be used effectively in Natural Language Processing, particularly for handling compound, specialized queries related to Indian parliamentary discussions.