RoBERTa-Based Multi-Feature Integrated BiLSTM and CNN Model for Ceramic Review Analysis

To address the limitation that the Robustly Optimized BERT Pretraining Approach (RoBERTa) may not effectively capture local dependencies and salient features within the text, we propose a feature fusion framework based on RoBERTa’s multi-output architecture. By feeding different outputs of RoBERTa into Convolutional Neural Networks (CNN) and Bidirectional Long Short-Term Memory (BiLSTM) networks, the model effectively captures both local static patterns and global contextual dependencies, thereby enhancing its ability to process complex textual information. Furthermore, an attention mechanism is incorporated to highlight critical features, improving the overall representation quality. To evaluate the effectiveness of the proposed model, we conducted comparative experiments on the Amazon, IMDB, SST- 2, and Ceramic datasets. The F1 scores achieved were 94.25%, 92.52%, 91.51%, and 81.32%, respectively, substantially outperforming existing models, particularly in handling domain-specific and nuanced texts. These results demonstrate that the proposed multi-feature fusion model significantly enhances sentiment classification performance.