Robustness-Enhanced Data-Free Quantization for Vision Transformer

Transformer has been a huge success in computer vision tasks, but its complex architecture and huge computational overhead limit the application of Vision Transformer (ViT) on edge devices. Recently, some works have focused on the field of data-free quantization of ViTs with good results. However, as they ignore the properties of ViT's non-smooth loss landscape, it leads to a quantization model that is susceptible to small perturbations. This paper proposes RE-ViT, a robustness-enhanced data-free quantization method for ViTs. RE-ViT utilizes Image Robustness Enhancement, which does interference with the input during data generation, to obtain synthetic samples with high semanticity and robustness. In the model quantization stage, contrast learning is introduced to alleviate the non-smooth loss landscape phenomenon of ViTs. A large number of experiments demonstrate the effectiveness of RE-ViT and it is shown to be superior to State-of-the-Art. RE-ViT establishes a new baseline for data-free quantization of ViTs.