Hierarchical Attention Networks with BERT Embeddings for Sentiment Analysis

We propose a novel approach for analysing sentiment by combining Hierarchical Attention Networks (HAN) with Bidirectional Encoder Representations from Transformers (BERT) embeddings to effectively handle long-form customer reviews. Traditional models often struggle to capture the hierarchical structure of lengthy text, where important sentiment clues exist at both word and sentence levels. By utilizing BERT's deep contextualized word representations within HAN's word-level and sentence-level attention mechanisms, our model is able to focus on the most relevant components of the text, improving sentiment classification accuracy. Applied to a large-scale Amazon reviews dataset consisting of 3.6 million training samples and 400,000 test samples, our model achieves an accuracy of 98.46% and a ROC AUC score of 98.68%, outperforming traditional models. These results demonstrate the effectiveness of this architecture in analyzing complex reviews, offering valuable insights for improving customer sentiment understanding and enabling more targeted marketing strategies.