Automated Sub-Feature Labeling Using Prompt-Based Pretrained Language Model

Abstract Many studies have been utilizing online user-generated data to draw product design implications via supervised and unsupervised approaches. While the supervised learning methods typically yield higher performance, they demand extensive data labeling tasks, consuming significant time and effort. This study proposes a framework that automatically labels online user data to address this limitation. The proposed framework consists of two pseudo-labeling mechanisms, keyword detection and prompting Pretraied Language Model (PLM). The first stage defines keywords for the target topic and then labels datasets by checking if the data contains these keywords. The second stage employs the PLM and labels datasets based on their context. Specifically, Prompting PLM adds a task-specific template at the end of the given text data (review) and predicts the masked token (label). This PLM-based approach serves as promising labeling candidates as they can make predictions without additional training data from the target domain. The suggested method was tested on a case study with real-world datasets. The study validates the effectiveness of this novel framework by comparing the pseudo-labeled results on smartphone sub-features to manual ground-truths. The results demonstrate that the new framework achieves F1 scores 28% and 14% higher than a baseline for screen and battery, respectively.