It's About Time: Incorporating Temporality in Retrieval Augmented Language Models

In this paper, we propose and evaluate TempRALM, a temporally aware retrieval-augmented language model with few-shot learning capabilities, which considers both the semantic and temporal relevance of retrieved documents in relation to a given query, rather than relying on semantic similarity alone. Our approach demonstrates up to <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\mathbf{7 4 \%}$</tex> improvement in performance over the baseline state-of-the-art retrieval-augmented language model ATLAS, and 32% improvement over a state-of-the-art commercial large language model augmented with retrieval. TempRALM achieves these improvements without requiring model pre-training, document index replacement, or other computationally intensive operations. Additionally, we introduce and evaluate TablePedia, a novel automated method for generating ground truth data for retrieval-augmented language models and temporal question-answering.