BerTaNER: A Tamil NER Framework Fine-Tuned with Adaptive Layerwise Learning Rate Decay (A-LLRD)

Named Entity Recognition (NER) is an essential component of the NLP domain. It enables one, amongst other things, to identify and categorize essential entities in text like individuals, places, and organizations. The breakthroughs seen in this field in recent years, made possible through transformer models such as BERT, are bringing promising results for high-resource languages like English. However, the applications for low-resource languages like Tamil are limited due to constraints created by lack of annotated datasets, and language-specific models. This paper introduces BerTaNER, a Tamil NER framework fine-tuned using Adaptive Layerwise Learning Rate Decay (A-LLRD)—a novel fine-tuning strategy, not previously applied to low-resource, morphologically rich languages like Tamil. Unlike traditional fixed-decay approaches, A-LLRD adapts the learning rate schedule to the training dynamics of each layer based on gradient norms, helping preserve deep linguistic features while optimizing shallow layers for task-specific adaptation. BerTaNER leverages both TamilBERT and mBERT, and is evaluated on the XTREME (PAN-X.ta) and IndicGLUE datasets. TamilBERT with A-LLRD achieves F1 scores of approximately 89.68 percent on XTREME and 89.12 percent on IndicGLUE, outperforming mBERT with A-LLRD, which scores about 85.2 percent on XTREME and 85.9 percent on IndicGLUE, respectively. These results demonstrate that A-LLRD enhances transformer finetuning in low-resource settings, offering a scalable approach for NER in morphologically rich and underrepresented languages.