Knowledge-Aware Prototypical Prompt Verbalizer: A New Approach for Few-Shot Text Classification

Lately, prompt tuning for pre-trained language models has proven to be highly effective in handling small sample classification tasks. This method involves framing the input text as a cloze-style question. For prediction purposes, the model relies on verbalizers to map output words to corresponding labels. Nevertheless, manually designed verbalizers are highly dependent on domain-specific prior knowledge. Moreover, automatically identifying suitable label words is particularly difficult in extremely in zero-shot situations. In this work, we propose an improved method for generating prototype label mappings. Specifically, during the generation of prototype vectors, we introduce a mechanism to incorporate sentence-level features into the prototype vectors. In zero-shot scenarios, we use a knowledge injection approach to incorporate potential knowledge contained in the labels for initialization. Furthermore, we employ data augmentation to enlarge the information content of the training samples. Experimental results on several low resource multi-class text classification datasets demonstrate the effectiveness of our method compared to other verbalizers construction approaches.