Automated Detection of Firm Social Media Response Strategies: A Multi-Label Classification Study of X-Based Customer Service Interactions

A considerable share of consumer-firm interactions today unfolds online. It is therefore indispensable for firms to understand the implications of their social media responses, especially when addressing negative consumer sentiment. A first step is to identify the response strategies firms deploy. This study is the first to incorporate large language models (LLMs) alongside deep learning to extract response strategies of service firms from their social media interactions with consumers. Employing the EmoTwiCS dataset, seven strategies are extracted from 5,299 interactions on X (formerly Twitter). We approach strategy identification as a multi-label classification problem since multiple strategies can be used in a single conversation. We compare deep learning models with various types of embeddings and LLMs to extract these strategies. Our results show that while LLMs with examples perform reasonably well, a custom-trained multi-layer perceptron model using Bag-of-Words representations performs best. This research offers valuable insights for future studies and organizations looking to analyze the effects of response strategies on service outcomes such as customer satisfaction, emotions, and the service recovery process.