Customer Complaint Classification with Large Language Models

Automatic classification of customer complaints is of great importance for companies to detect customer problems early, optimize service processes and increase customer satisfaction. Customer complaint data provided as textual data consists of very insightful information in terms of addressing the problems encountered. In this study, deep learning-based approaches (LSTM, CNN), transformer-based models (BERT, RoBERTa) and large language models (GPT-2, Llama3.1-8B) are compared using two customer complaint datasets with different class distributions. The results show that LLMs achieve higher accuracy and F1 scores in both training and test phases. In particular, the fine-tuning process applied to the large language models enabled them to be tailored to the problem and contributed to the higher performance of LLMs compared to other models.