Quantization of Tiny MLP-Mixer and Its Evaluation

MLP-Mixer is a deep learning-based image classifier model that does not have any convolutional or self-attention mechanism, and is attracting attention for its competitiveness of accuracy and throughput against conventional convolutional neural networks (CNNs) and state-of-the-art models such as Vision Transformer (ViT). This study proposes a quantization method for MLP-Mixer aimed at hardware circuit implementation, and evaluate its effectiveness through experiments. The results demonstrate that MLP-Mixer can be trained with low precision to a certain extent, and when represented using 4-bit and 8 -bit integers, its accuracy is comparable to that of a highprecision model that uses 32-bit floating-point numbers.