Sentiment Analysis on Indonesian-Javanese-English Code-Mixed Texts via Parameter-Efficient Fine-Tuning and Selective Layer Unfreezing

This study presents a parameter-efficient fine-tuning framework for sentiment analysis on Indonesian-Javanese-English code-mixed texts using Transformer-based models. Motivated by the growing prevalence of multilingual expression on social media and the challenges of limited computational resources in low-resource settings, we explore the effectiveness of Low-Rank Adaptation, a Parameter-Efficient Fine-Tuning technique, in combination with Selective Layer Unfreezing. These strategies are designed to reduce the number of trainable parameters while retaining strong model performance. To evaluate this approach, we fine-tuned five pre- trained Transformer models: IndoBERT, IndoBERTweet, M- BERT, XLM-RoBERTa-base, and XLM-RoBERTa-large. Our experiments showed that Low-Rank Adaptation was particularly effective for smaller, domain-adapted models such as IndoBERTweet, which achieved the highest F1 score of 91.12%. The combination of Low-Rank Adaptation and Selective Layer Unfreezing further improved performance in larger multilingual models, such as XLM-RoBERTa large, increasing the F1 score from 80.65 to 86.29%. Across all configurations, the number of trainable parameters was kept under one percent. These findings demonstrate that the integration of Parameter-Efficient Fine-Tuning and Selective Layer Unfreezing provides an effective and efficient solution for sentiment analysis in code-mixed and computationally constrained environments.