Preserving Label Correlation for Multi-label Text Classification by Prototypical Regularizations

Multi-label text classification (MLTC) assigns multiple labels to a sentence, with the key challenge being capturing label correlations. Existing models prioritize leveraging correlations but often overlook overfitting, while plug-and-play regularization methods fail to preserve correlations effectively. In this paper, we distinguish two types of label correlations: explicit co-occurring correlations and implicit semantic correlations, and propose regularizations on prototypical label embeddings for correlation preservation. Specifically, we first generate the prototypical embedding of multiple co-occurred labels as an intermediate. We then apply a prototypical regularization on the distance between the sentence embedding and corresponding prototypical embedding to alleviate the over-alignment issue caused by binary cross entropy loss and facilitate explicit correlation preservation. We finally extend the vanilla Mixup, which solely mixes multi-hot labels, on prototypical embedding mixing to promote implicit correlation preservation. Empirical studies show the effectiveness of our regularization methods.