Robust Neural Information Retrieval: An Adversarial and Out-of-Distribution Perspective

Recent advances in neural information retrieval models have significantly enhanced these models’ effectiveness across information retrieval tasks. The robustness of these models, which is essential for ensuring their reliability in practice, has also garnered significant attention. With a wide array of research on robust information retrieval being published, we believe it is the opportune moment to consolidate the current status, glean insights from existing methodologies, and lay the groundwork for future development. Robustness of information retrieval is a multifaceted concept and we emphasize the importance of robustness against performance variance, out-of-distribution scenarios, and adversarial attacks. With a focus on out-of-distribution and adversarial robustness, we dissect robustness solutions for dense retrieval models and neural ranking models, respectively, recognizing them as pivotal components of the neural information retrieval pipeline. We provide an in-depth discussion of methods, datasets, and evaluation metrics, shedding light on challenges and future directions in the era of large language models. To accompany this survey, we release three additional resources: (1) a curated list of publications related to robust information retrieval, (2) a tutorial based on this survey, and (3) a heterogeneous benchmark for robust information retrieval, BestIR, that collects all known datasets for evaluating information retrieval systems for robustness. We hope that this study provides useful clues for future research on the robustness of information retrieval models and helps to develop trustworthy IR systems.