Parameter Efficient Fine-tuning using Low-Rank Adaptation for Emotion Classification in Indonesian Texts

The rapid expansion of social media has created a dynamic platform in which individuals regularly share their thoughts and emotions, leading to the generation of extensive textual data. This vast amount of data offers both an opportunity and a challenge for automatic emotion recognition, as effective utilization requires advanced models capable of accurately classifying emotions across varied linguistic contexts. In this study, we conducted a comparative analysis of emotion classification using Transformer models, specifically IndoBERTweet, Multilingual BERT, and XLM-RoBERTa, with a focus on full-parameter fine-tuning and Parameter-Efficient Fine-Tuning (PEFT) using Low-Rank Adaptation (LoRA). The experiments demonstrated that although PEFT with LoRA significantly reduces the number of trainable parameters, it can still achieve competitive performance compared to full-parameter fine-tuning, with XLM-RoBERTa (large) showing the highest precision, recall, and Macro-F1 score. However, the underperformance of M-BERT when using PEFT highlights the complexity of optimizing these models, indicating that not all models are equally suited to this fine-tuning approach. These findings suggest that PEFT methods such as LoRA can be effective alternatives to full-parameter fine-tuning, particularly for larger models. Future research could compare Transformer models across different languages and emotion classification tasks, explore alternative PEFT methods beyond LoRA, investigate the effects of LoRA on different Transformer layers, and examine hybrid fine-tuning approaches for improved efficiency and performance, especially for models such as M-BERT.