Parameter-Efficient Fine-Tuning Methods for Multilingual Named Entity Recognition

Named Entity Recognition (NER) is a crucial task in natural language processing that enables machines to better understand human languages. The multilingual NER task is challenging due to the significant syntactic and lexical variations across languages. While large pre-trained models like XLM-R have shown effectiveness on multilingual NER, they require substantial computational resources. Parameter-Efficient finetuning (PEFT) methods have recently gained attention as they can achieve comparable performance to full fine-tuning while using only a fraction of the parameters. In this study, we evaluate the effectiveness of three parameter-efficient fine-tuning (PEFT) approaches: bottleneck adapters, language adapters, and LoRA, for multilingual named entity recognition (NER) based on the XLM-R model. Our results demonstrate that adapter and LoRA tuning are highly effective, yielding an average 1.8% improvement on macro-F1 while using less than 1% of the baseline model’s parameters. Furthermore, we find that PEFT enables robust transfer learning, boosting performance on low-resource languages by leveraging high-resource base models. However, cross-lingual transfer is impacted by syntactic and textual similarity between languages, necessitating richer data for dissimilar languages. Our findings highlight PEFT’s potential for multilingual NER and provide valuable insights for future work.