Code-Mixed Sentiment Analysis on Indonesian-Javanese-English Text Using Transformer Models

Sentiment analysis plays a crucial role in under-standing opinions and societal trends, but it encounters chal-lenges when dealing with code-mixed text, which blends multiple languages. While most studies focus on bilingual code-mixing, trilingual mixing is common yet underexplored in sentiment analysis research. This study addresses this gap by examining sentiment analysis in code-mixed Indonesian-Javanese-English text from Twitter data using Transformer models. The dataset was annotated by two annotators fluent in Indonesian, Javanese, and English. We evaluated the annotation results using Cohen's kappa and achieved high agreement, with a score of 0.9767. Furthermore, we performed hyperparameter tuning using Optuna to obtain the best hyperparameters for fine-tuning tasks using the Transformer models. Our experiments with pre-trained monolingual models, such as IndoBERT and IndoBERTweet, outperformed the pre-trained multilingual models (MBERT, DistilMBERT, and XLM-RoBERTa) in precision, recall, and F1 scores (IndoBERTweet achieved the highest F1 score of 94.14). Future research could explore translating code-mixed sentences into a single language and parameter-efficient tuning methods to further enhance the accuracy and efficiency of sentiment analysis.