Hybrid Deep Learning Approaches for Assamese Part-of-Speech Tagging Using BIS Tag Set

In this study, a parts-of-speech (POS) tagger designed exclusively for the Assamese language, utilizing advanced neural network architectures such as Long Short-Term Memory (LSTM) and Bidirectional LSTM (Bi-LSTM) was proposed. The primary objective is to accomplish high-accuracy POS tagging for Assamese text using the Bureau of Indian Standards (BIS) tagset. A large Assamese corpus was created and annotated with BIS-compliant POS tags. After painstaking data preprocessing to clean and tokenize the text, words and tags were turned into numerical representations. The neural network architecture includes an embedding layer with pre-trained word embeddings, an output layer for POS tag prediction, and several LSTM and Bi-LSTM layers that recognize sequential patterns. The models include LSTM at the word and character level and LSTM-CRF and Bi-LSTM-CRF variants. The LSTM-word level model achieved 87.39% accuracy, LSTM-CRF word level 89.13%, Bi-LSTM word level 91.83%, Bi-LSTM-CRF word level 93.23%, Bi-LSTM character level 96.17%, and Bi-LSTM-CRF word level 94.24%. This work outperforms prior Assamese POS tagging efforts and provides useful insights for related languages.