A Comparative Study: Leveraging BERT and K-fold Cross Validation for UPOS Tagging in Bengali

Universal Dependency (UD) infrastructure-based Part-of-Speech tagging also called UPOS tagging, holds critical importance for the development of higher-level Natural Language Processing (NLP) applications in Bengali. Due to its limited resources and lack of exploration, this study represents a manually created Bengali UPOS dataset and a comparative analysis utilizing recurrent-based deep learning models and transformer-based techniques to enhance tagging reliability. For the creation of the dataset, the information was gathered from news, tales, Wikipedia, and various subject areas in order to preserve the Bengali language’s prevalence, and we have painstakingly compiled over 2200 sentences with more than 32000 tokens into a treebank. In order to increase UPOS tagging accuracy, we adopt a transformer-based technique with multilingual BERT (mBERT) for our dataset and initially yield an 84% accuracy after fine-tuning it. Our BERT model shows a significant performance gain when k-fold cross-validation (k=5) is applied, and the accuracy dramatically improves to 92.65% on average. We also compare our approach against other recurrent-based models (e.g., LSTM, BiLSTM) that are also applied to the same dataset, and the efficacy of a transformer-based model named BERT for morphologically rich and low-resource languages like Bengali is demonstrated. The enhanced accuracy and vast dataset in Bengali UPOS tagging could significantly impact NLP applications like dependency parsing, grammar checkers, and machine translation enhancement.