BERT-MLTKE: A Multi-task Deep Learning Framework for Keyphrase Extraction in Social Media

Keyphrase extraction is a critical task in natural language processing (NLP) aimed at automatically identifying and extracting essential phrases from a given text. Traditionally, keyphrase extraction has been extensively studied in the context of structured text sources such as articles, documents, and web pages. With the growth of social media platforms like Twitter (X), the demand for effective keyphrase extraction has become increasingly important in not only public opinion and sentiment understanding but also facilitating real-time trend identification and social governance. Keyphrase extraction from social media presents unique challenges due to the informal, unstructured, and highly diverse nature of the data. Existing methods mainly rely on either unsupervised (statistical based) approaches or supervised (deep learning based) techniques, each of which has its own advantages and disadvantages. In order to leverage the strengths of both supervised and unsupervised learning, we propose BERT-MLTKE, a BERT-based multi-task framework designed to significantly enhance keyphrase extraction performance on social media data. Our framework fine-tunes the pre-trained BERT model to effectively capture both contextual information and syntactic structures within sentences. Instead of using traditional unsupervised approaches of extracting candidate phrases and ranking them based on statistical metrics, we introduce a multi-task module with supervised deep learning models that addresses both keyword identification and keyphrase extraction as two progressive tasks, enabling the framework to share essential information while tackling multiple subtasks concurrently. During the implementation of deep learning models, we used several techniques that contribute to model stability. For further enhancement in framework’s performance, we conducted several comparative experiments, optimizing both the architecture of BERT-MLTKE and deep model candidates selection in embedding and feature transformation layers. The results of these experiments were thoroughly analyzed, and we offer possible explanations for the observed variations for different deep models in task performance. We also propose strict evaluation metrics to ensure a more precise assessment of the experimental results in an objective manner.