A Study of Backdoor Attacks on Data Distillation for Text Classification Tasks

Data distillation is a technique to reduce the training data for machine learning models, by which a large training dataset can be reduced to a smaller, more streamlined dataset while retaining the key information of the original dataset and the model's learning capability. Previous research on the security of data distillation has only been done on image data, and the security of text data distillation has not yet been studied. In this paper, we conduct a systematic research on backdoor attack methods for text data in the process of data distillation. Specifically, this paper embeds a backdoor and implements a backdoor attack before and during data distillation of text data. The experimental results show that, relative to embedding the backdoor before the data distillation operation, embedding the backdoor during the distillation process can make the backdoor information more effectively retained in the distilled data, and can achieve a 100% success rate of the attack under the premise of guaranteeing good concealment. These results show that although the data distillation technique can improve the efficiency of model training, its potential security risk should not be ignored.