Proto-Att-FSL: enhanced prototypical network for cross-domain few-shot airline sentiment classification

In literature, the challenge of having limited labeled data is indeed significant, especially in fields of applying machine learning and deep learning algorithms. Several state-of-the-art methods, particularly deep learning techniques, are often data-hungry, as they require large labeled datasets to train effectively and to generalize well to new, unseen data. To address this issue, we have adopted metric learning, specifically focusing on few-shot learning methods, and leveraging transfer learning from pre-trained transformers. We have proposed a multi-attention-based prototypical network (Proto-Att-FSL), and our observations indicate that this proposed method offers superior class representation compared to approaches that do not utilize an attention-based prototypical network. For our experimental setup, we utilized an Airline sentiment analysis dataset. To maintain the data in close proximity, we divided the dataset into two distinct domains based on specific keywords. This approach allowed us to test both the proposed methods and their baseline counterparts within each domain as well as across different domains. We conducted experiments with configurations of 10, 30, and 50 shots, along with attention heads set to 1, 4, 6, and 8. Our observations revealed that the 50-shot, 4-head attention setup surpassed the baseline model’s performance and achieved 82.3% accuracy in the in-domain tests and the 30-shot. 1-head attention setup achieved 79.6% accuracy in the cross-domain evaluations. This result highlights the substantial impact of multi-attention mechanisms in prototypical networks for few-shot learning.