Exploring the Power of Vision Transformer (ViT) and MLP-Mixer in Image Classification: A Comparative Study and Prospects for ViT-Mixer Architecture

For image classification tasks, the convolutional neural network (CNN) has so far been the automatic choice. However, the revolutionary triumph of the Transformer architecture in NLP prompted researchers to apply it to computer vision, which eventually resulted in the opening of a new window called Vision Transformer (ViT). ViT showed an equivalent result as CNN and sometimes even better than CNN when trained with a large dataset. More recently, some researchers showed that with MLP-mixer alone, it is possible to get almost the same accuracy as ViT in image classification but using fewer resources and a lower computational complexity than ViT. The paper discusses the recent success of the ViT and MLP-Mixer (Mixer) architectures in image recognition and discusses their performance on multiple datasets after implementing both of them using Python. ImageNet and CIFER-10 ViT performed 84.06% and 99.01%, respectively, which is 1.04% better in terms of accuracy than that of MLPMixer overall. However, Mixer uses 20% less computational resources. Once we changed some encoder layers of ViT with MLP, even though the result was 0.56% lower than that of pure ViT, the required resource came down by 9%. Therefore, it's possible to make a tradeoff between computational complexity and accuracy by replacing some layers of ViT with an MLP mixer. The prospect of mixing these two architectures to make a ViT-Mixer architecture that can reduce the computational burden of ViT while maintaining its accuracy is further covered in the paper.