Cross-domain sentiment classification in Slovak: a comparative analysis of transformer-based models

Background Three-class sentiment classification (positive, negative, neutral) is a critical aspect of various natural language processing (NLP) applications. However, for the Slovak language, it presents significant challenges, primarily due to limited linguistic resources and less research attention compared to major languages. Currently, there is no sentiment classifier for Slovak capable of effectively classifying text across multiple domains. This study presents the comprehensive evaluation of multi-domain sentiment classification for the Slovak language, addressing a significant gap in existing NLP research. Methods We compiled and preprocessed five balanced sentiment datasets (positive, neutral, negative), combining Slovak and machine-translated Czech resources. Five transformer-based language models were fine-tuned using stratified 10-fold cross-validation: Cross-lingual Language Model-Robustly optimized Bidirectional Encoder Representations from Transformers approach (XLM-RoBERTa) (Base), SlovakBERT, DistilBERT, and two compact Slovak-specific models—Slovak Byte-Pair Encoding Baby Language Model (SK_BPE_BLM) and its morphological variant Slovak Morphological Baby Language Model (SK_Morph_BLM). Model performance was assessed using accuracy, precision, recall, and F1-score. To determine domain effects and model differences, we performed Friedman tests and Nemenyi post-hoc comparisons. Per-class F1-scores, training dynamics, and computational efficiency were also analyzed. Results Sentiment classification performance varied significantly across domains for all models (Friedman p < 0.005), confirming strong domain dependency. SlovakBERT achieved the highest mean F1-scores in the banking, social media, and politics domains, while SK_Morph_BLM led in the movie domain, and XLM-RoBERTa performed best in the reviews domain. The Slovak-specific models exhibited the greatest stability, with SK_Morph_BLM achieving the smallest neutral–positive F1 disparity (ΔF1 ≈ 0.06). SlovakBERT and SK_Morph_BLM provided the most balanced per-class performance. Computationally, SK_BPE_BLM and SK_Morph_BLM were the fastest (≈532 s/fold), while XLM-RoBERTa required the most time (≈1,673 s/fold). Overall, SlovakBERT delivered the strongest cross-domain performance, whereas the Slovak-specific models offered the best efficiency–accuracy trade-off.