Exploring the Efficacy and Bias of Sentiment Analysis Tools in Customer Reviews

Abstract Sentiment analysis is an important technique for understanding public sentiment in customer feedback. This study aims to compare the effectiveness and reliability of BERT and VADER in analysing customer reviews from an e-commerce platform. We analyzed mean sentiment scores and standard deviations for each tool, using Spearman’s Rho to assess correlations between VADER and BERT results due to non-normal score distributions. Our findings reveal that VADER predominantly categorizes reviews as Neutral, with limited Positive and Negative classifications, while BERT shows a bias towards Negative sentiments with some Positive and no Neutral classifications. A weak positive correlation indicates limited agreement between the two tools. Additionally, neither tool showed a strong correlation with actual review ratings, highlighting challenges in accurately capturing nuanced sentiments. The study discusses potential biases inherent in each tool’s methodology – VADER’s lexicon-based approach may oversimplify sentiments, and BERT’s deep learning architecture may introduce a negative bias. These findings suggest that combining VADER’s efficiency with BERT’s contextual sensitivity – for instance, through hybrid models – can mitigate individual tool limitations and offer a more robust sentiment analysis framework for e-commerce businesses. By integrating these insights, businesses can optimize customer feedback systems, refine marketing strategies, and improve product development processes