ReGAT-BERT: Transformer-Graph Fusion for Dynamic Reranking

Effective and precise passage retrieval is crucial in contemporary information retrieval systems, particularly in light of the exponential expansion of digital document collections. Conventional approaches relying on lexical matching frequently struggle to capture semantic subtleties, often overlooking passages that are semantically pertinent but lexically divergent from user queries. Recent advancements in transformer-based models, such as BERT and its variants, have substantially enhanced retrieval performance by facilitating a deeper semantic comprehension. Nonetheless, the computational demands of these models present scalability challenges when applied to large datasets. In this work, we propose an innovative passage retrieval framework that seamlessly integrates lexical-semantic relevance with contextual information, markedly boosting retrieval precision and coherence. Our methodology unfolds in three stages: first, a Cross-Encoder BERT model assesses semantic relevance by analyzing fine-grained interactions between queries and passages. Second, a Bi-Encoder architecture generates initial embeddings, which are subsequently refined using a Graph Attention Network (GAT) to incorporate structural and contextual relationships among passages. Third, a dynamic reranking mechanism improves ranking by merging semantic relevance scores with contextual similarity evaluations, ensuring that retrieved passages are both contextually coherent and semantically precise. Evaluations conducted on benchmark datasets against state-of-the-art baselines reveal that our hybrid approach enhances retrieval accuracy and coherence. Furthermore, by integrating graph-based embedding refinements, our model mitigates the scalability constraints typically associated with Cross-Encoder architectures.