Evaluation of LLMs, BERT, and Ensemble Techniques for Analyzing Online Vaccine Sentiment

The widespread use of social media platforms during the COVID-19 pandemic has highlighted the importance of understanding public sentiment towards vaccines. This paper presents an evaluation of large language models (LLMs), a fine-tuned BERT model, and ensemble methods for analyzing social media posts for vaccine sentiment. We introduce a novel dataset comprising annotated social media posts and evaluate the performance of various models in sentiment classification. Our results demonstrate that LLMs can accurately classify sentiments, open-source LLMs have approached parity with closed-source LLMs, ensemble models can provide marginal improvements, and fine-tuned SLMs can approach the performance of LLMs at much smaller footprint and running costs, making them feasible in many resource-constrained environments.