Does Bigger Mean Better? A Domain-Aware Comparison of Lexicon-Based, TF-IDF, and Transformer Models for Sentiment Analysis

Sentiment analysis systems range from training-free lexicons to compact supervised models and fine-tuned transformer encoders, but fair comparison requires identical test examples and explicit computational accounting. (1) Background: We reassess three sentiment-classification paradigms under matched binary-label evaluation on IMDb movie reviews and Sentiment140 tweets. (2) Methods: After duplicate and label audits, VADER, TF–IDF with validation-tuned Logistic Regression, and three-epoch GPU-fine-tuned DistilBERT were evaluated on the same 5065 IMDb and 10,000 Sentiment140 held-out records within each corpus. Accuracy, macro-F1, confusion matrices, paired McNemar tests, bootstrap intervals, training time, and inference throughput were computed from saved predictions and execution logs. (3) Results: DistilBERT reached 92.00% accuracy on IMDb and 83.41% on Sentiment140, compared with 91.06% and 78.77% for Logistic Regression and 69.83% and 66.54% for VADER. DistilBERT exceeded Logistic Regression by 0.95 percentage points on IMDb (paired 95% CI 0.08–1.76; p=0.0351) and by 4.64 points on Sentiment140 (CI 3.85–5.41; p<0.001), but required GPU fine-tuning and longer measured training. (4) Conclusions: The relative advantage of a transformer depends on the dataset and available computation. Logistic Regression remains a compact, accurate alternative; results are restricted to these two within-domain configurations and do not demonstrate cross-domain transfer or domain causation.