Named Entity Recognition in Assamese Language using two separate models: BiLSTM and BERT

Named Entity Recognition (NER) is a tool based on principles of Artificial Intelligence (AI) and Natural Language Processing (NLP) for automatically tagging Named Entities from unstructured text. In the realm of Natural Language Processing (NLP) applications, Named Entity Recognition (NER) holds significance as it involves the crucial task of identifying and categorizing proper nouns into classes such as person, location, organization, and miscellaneous. While considerable progress has been made in widely spoken languages like English and other European languages, resulting in higher accuracy rates, the task of NER in Indian languages prove to be challenging due to limited resources. This study explores the implementation of NER in Assamese using two separate approaches: BiLSTM and BERT. The proposed methodology achieves an accuracy of 31%in the BiLSTM model. While using BERT, which is a pretrained model, fine-tuned for Assamese, we achieved a precision of 81.5% and F1- score of 0.383. Our comparative analysis shows that both models are effective for NER in a resource-scarce language like Assamese, but BERT performs better overall in recognizing entities. This suggests that BERT could play a key role in improving NER techniques for underrepresented languages.