Transformer Based Approaches for Sentiment Analysis on Restaurant Domain

Abstract: This project addresses the critical challenge of accurately analysing sentiment in restaurant reviews, where traditional machine learning methods struggle with linguistic nuances. Existing solutions, which are frequently basic ML or older deep learning models, typically lack the capacity to properly address class imbalance and capture deep contextual knowledge. To address these limitations, the study implements and evaluates two advanced transformer-based models: fine-tuning RoBERTalarge with Focal Loss, which focuses on learning from difficult examples and underrepresented classes, and a hybrid DistilBERT-CNN-BiLSTM model with fast gradient method, which combines efficient contextual embedding, local feature extraction, and sequential pattern recognition. These models aim to improve sentiment classification performance by making use of transformer-based architectures and modifying loss functions. Aspect-based sentiment analysis integration and the implementation of models for real-time feedback systems are examples of future development.