Transfer Learning Approach for Sentiment Analysis in Low-Resource Austronesian Languages Using Multilingual BERT

Sentiment analysis for low-resource languages, particularly Austronesian languages, remains challenging due to the limited availability of annotated datasets. Traditional approaches often struggle to achieve high accuracy, necessitating strategies like cross-lingual transfer and data augmentation. While multilingual models such as mBERT offer promising results, their performance heavily depends on fine-tuning techniques. This study aims to improve sentiment analysis for Austronesian languages by fine-tuning mBERT with augmented training data. The proposed method leverages cross-lingual transfer learning to enhance model robustness, addressing the scarcity of labeled data. Experiments were conducted using a dataset enriched with augmentation techniques such as back-translation and synonym replacement. The fine-tuned mBERT model achieved an accuracy of 92%, outperforming XLM-RoBERTa at 91.41%, while mT5 obtained the highest accuracy at 99.61%. Improvements in precision, recall, and F1-score further validated the model’s effectiveness in capturing subtle sentiment variations. These findings demonstrate that combining data augmentation and cross-lingual strategies significantly enhances sentiment classification for underrepresented languages. This study contributes to the development of scalable Natural Language Processing (NLP) models for Austronesian languages. Future research should explore larger and more diverse datasets, optimize real-time implementations, and extend the approach to tasks such as Named Entity Recognition (NER) and machine translation. The promising results underscore the importance of integrating robust transfer learning techniques with comprehensive data augmentation to overcome challenges in resource-limited NLP scenarios