Factual Knowledge in Language Models: Robustness and Anomalies under Simple Temporal Context Variations

This paper explores the robustness of language models (LMs) to variations in the temporal context within factual knowledge.It examines whether LMs can correctly associate a temporal context with a past fact valid over a defined period, by asking them to differentiate correct from incorrect contexts.The LMs' ability to distinguish is analyzed along two dimensions: the distance of the incorrect context from the validity period and the granularity of the context.To this end, a dataset called TimeStress is introduced, enabling the evaluation of 18 diverse LMs.Results reveal that the best LM achieves a perfect distinction for only 11% of the studied facts, with errors, certainly rare, but critical that humans would not make.This work highlights the limitations of current LMs in temporal representation.