Diffusion Model Empowered Efficient Data Distillation Method for Cloud-Edge Collaboration

The application of AI-generated models demands substantial amounts of data, which not only increases training time and memory consumption but also poses challenges to computation and communication loads in cloud-edge collaborative environments. Especially when edge devices have limited resources and network bandwidth is constrained, efficiently handling large-scale data in cloud-edge collaboration becomes a critical issue. Data distillation compresses large datasets into smaller, synthetic datasets, reducing transmission and computation overhead while maintaining model performance. However, existing data distillation methods face two main challenges: 1) improving accuracy without compromising distillation efficiency, and 2) the distilled dataset may retain backdoor attack triggers. To address these challenges, we propose an efficient data distillation method based on diffusion models, tailored for cloud-edge collaborative environments. We design a diffusion model training mechanism based on Kullback-Leibler (KL) divergence and contrastive loss to effectively distill synthetic datasets and enhance their accuracy, thereby reducing the data storage requirements on edge devices and alleviating communication burden with the cloud. Additionally, we introduce a dynamic channel weighting method based on an adaptive attention mechanism to retain essential features during the distillation process, improving model adaptability in cloud-edge collaborative environments. For the first time, We propose an optimization method based on diffusion model classification loss to realize backdoor attack in data set distillation in edge environment. Experimental results demonstrate that our method improves the accuracy of distilled datasets by 2.8% to 11.7% compared to traditional DNN data distillation methods and by 0.9% to 4.1% compared to existing diffusion model-based data distillation methods. Additionally, our attack methodology achieves backdoor attacks while maintaining an accuracy loss of less than 5% on the original model when the triggers are not visible.