Advancing continual lifelong learning in neural information retrieval: Definition, dataset, framework, and empirical evaluation

Continual learning refers to the capability of a machine learning model to learn and adapt to new information, without compromising its performance on previously learned tasks. Although several studies have investigated continual learning methods for neural information retrieval (NIR) tasks, a well-defined task definition is still lacking, and it is unclear how typical learning strategies perform in this context. To address this challenge, a systematic task definition of continual NIR is presented, along with a multiple-topic dataset that simulates continuous information retrieval. A comprehensive continual neural information retrieval framework consisting of typical retrieval models and continual learning strategies is then proposed. Empirical evaluations illustrate that the proposed framework can successfully prevent catastrophic forgetting in neural information retrieval and enhance performance on previously learned tasks. The results also indicate that embedding-based retrieval models experience a decline in their continual learning performance as the topic shift distance and dataset volume of new tasks increase. In contrast, pretraining-based models do not show any such correlation. Adopting suitable learning strategies can mitigate the effects of topic shift and data augmentation in continual neural information retrieval. • A continual learning framework is proposed and implemented for continuous neural information retrieval tasks. • A dataset called Topic-MSMARCO, containing diverse topics for continual neural information retrieval tasks, is provided. • The best continual learning strategies for diverse retrieval models are identified. • Empirical evaluations demonstrate the impact of topic shift and data volume on continual information retrieval.