SentiMatrix: Parameter-Efficient Fine-Tuning of Encoder-Based Transformers for Multidimensional Sentiment Analysis

Abstract Transformer-based architectures have established state-of-the-art benchmarks across Natural Language Processing (NLP) tasks; however, the computational overhead of full fine-tuning remains a significant barrier to scalable deployment. This paper presents SentiMatrix, a systematic evaluation of Low-Rank Adaptation (LoRA)-based Parameter-Efficient Fine-Tuning (PEFT) across four sentiment analysis paradigms: Intent-based, Aspect-based, Fine-grained, and Emotion detection, spanning seven benchmark datasets under a consistent three-stage protocol: inference using task-specific pretrained baseline models (i.e., HuggingFace checkpoints previously fine-tuned on the corresponding benchmarks, serving as informed upper-bound references rather than cold-start zero-shot baselines), Full Fine-Tuning (FFT) as a performance upper bound, and LoRA-based adaptation to quantify the efficiency-performance trade-off. Adaptive Low-Rank Adaptation (AdaLoRA) is benchmarked as a direct comparator. LoRA achieves competitive performance while reducing trainable parameters by up to 99.8%, with lower GPU memory usage and training time in most evaluated settings. It achieves 93.28% accuracy on SST-2, 80.74% on combined Laptop+Restaurant ABSA, 67.63% on E-Commerce 5-class, and 93.22% on emotion detection, while closely approaching FFT on E-Commerce 5-class (68.47%). The results demonstrate an effective efficiency-performance trade-off across diverse sentiment analysis tasks. Ablation experiments examine the effects of adaptation strategy, model capacity, and label granularity. The source code and datasets are available at https://github.com/ELTE-DSED/senti-matrix .