BabelEdits: A Benchmark and a Modular Approach for Robust Cross-lingual Knowledge Editing of Large Language Models

With Large Language Models (LLMs) becoming increasingly multilingual, effective knowledge editing (KE) needs to propagate edits across languages.Evaluation of the existing methods for cross-lingual knowledge editing (CKE) is limited both w.r.t.edit effectiveness: benchmarks do not account for entity aliases and use faulty entity translations; as well as robustness: existing work fails to report on downstream generation and task-solving abilities of LLMs after editing.In this work, we aim to (i) maximize the effectiveness of CKE while at the same time (ii) minimizing the extent of downstream model collapse due to the edits.To accurately measure the effectiveness of CKE methods, we introduce BABELEDITS, a new CKE benchmark covering 60 languages that combines high-quality multilingual synsets from BabelNet with marker-based translation to ensure entity translation quality.Unlike existing CKE benchmarks, BABELEDITS accounts for the rich variety of entity aliases within and across languages.We then propose BABEL-REFT, a modular CKE approach based on representation fine-tuning (ReFT) which learns entity-scope ReFT modules, applying them to all multilingual aliases at inference.Our experimental results show that not only is BABEL-REFT more effective in CKE than state-of-theart methods, but, owing to its modular design, much more robust against downstream model collapse when subjected to many sequential edits. 1