DistilBERT Hotel Rating Prediction Model Based on an Ensemble Learning Framework

The swift advancements in artificial intelligence and natural language processing have significantly impacted the hospitality industry by enabling more precise sentiment analysis and recommendation systems. This study presents a novel hotel rating prediction model leveraging DistilBERT, a distilled version of BERT, within an ensemble learning framework. DistilBERT retains 97% of BERT's performance while being 60% faster and 40% smaller, making it ideal for real-time applications in resource-constrained environments. Our ensemble model combines two DistilBERT models, one processing review titles and the other processing review texts, to capture comprehensive semantic information from both sources. The dataset used comprises approximately 10,000 hotel reviews, preprocessed to ensure quality and consistency. Through rigorous training and evaluation, including the use of cross-validation and random search for hyperparameter tuning, our model demonstrated superior performance compared to traditional machine learning models and other pre-trained language models such as BERT, RoBERTa, and ALBERT. The DistilBERT ensemble model achieved high accuracy, precision, recall, and F1 score, showcasing its effectiveness in handling complex text classification tasks. This research provides valuable insights for the hospitality industry, enabling better prediction of customer satisfaction and enhancing service quality. Future work could explore additional features, multi-lingual capabilities, and further optimization to improve model performance.