VBiT: Enhancing Vision Transformers with Binary Techniques for Image Classification

In the field of artificial intelligence, modern pre-trained transformers have significantly advanced the technological frontier. However, the increasing number of parameters and computational complexity make their deployment in resource-constrained environments increasingly challenging. This paper proposes an innovative method, VBiT, which combines Vision Transformers (ViT) and Binary Transformers (BiT) to significantly reduce model size and computational requirements through binarization techniques, thereby enhancing deployment efficiency. First, we introduce how the ViT model divides images into patches and converts them into tokens, which are then processed by the transformer. Next, it delves into the principles of BiT binarization, including mean subtraction, elastic binarization functions, and their optimized scaling factors, providing an efficient multi-step distillation method to improve model accuracy. Experimental results demonstrate that VBiT performs excellently on standard image categorization datasets such as ImageNet, CIFAR-10, CIFAR-100, COCO, and PASCAL VOC, maintaining high accuracy even under extreme low-bit quantization conditions. This approach not only offers a new perspective on the practical application of binary transformers but also showcases the potential of combining ViT and BiT in image categorization tasks.