A Prompting Framework for GPT-Based Twitter/X User Classification in the Context of Disasters

During disasters, social media users from diverse social groups share information, concerns, and perspectives on X (formerly Twitter), amassing critical disaster resilience knowledge. Effective user classification is essential for facilitating user-centered research across different groups and improving the relevance and quality of social media data mining. This study examines the feasibility of leveraging a large language model (LLM), generative pretrained transformer (GPT)-4o Mini, for the social role-related classification of Twitter/X users in the disaster contexts. A three-step social group classification (TSSGC) prompting framework was proposed, incorporating several prompting and improvement methods to enhance GPT’s reasoning capabilities. The performance of the TSSGC was evaluated across different classification scenarios, including stakeholder sectors to which users belong, the industries they are affiliated with, and the disaster-related responsibilities they take. Comparative experiments were conducted to evaluate the performance of the TSSGC against several widely used machine-learning and prompting methods. The results demonstrate the effectiveness and applicability of the TSSGC in categorizing users into various social groups. This research contributes to the body of knowledge by proposing a prompting framework that leverages LLMs to classify Twitter/X users into disaster-related social groups based solely on usernames and descriptions. By supporting user classification using minimal user information, the proposed approach minimizes computational demands and reliance on extensive data, thus enhancing the scalability and efficiency of large-scale social media processing in time-sensitive disaster settings.