A Novel Mixed-Precision Quantization Approach for CNNs

Model size and inference speed have brought about major challenges for the deployment of Convolutional Neural Networks (CNNs) in many applications. An effective approach to address this issue is model quantization, which achieves network compression and inference speedup by reducing the parameters bit-width during inference. Mixed-precision quantization assigns different quantization precision to different layers of a CNN. However, due to accuracy degradation, determining the precision of each layer remains a challenge. In this paper, we propose an efficient mixed-precision search method that tackles this problem. It first utilizes the average approximation trace of the Hessian matrix to determine the sensitivity of each convolutional layer to quantization. It then establishes the optimization objective of the mixed-bit-width search network, and combines the cosine distance of the outputs of the two neighboring layers in the search process in order to construct a micro-minimizable combination of mixed-bit-widths for the super-net search. It finally adds the conditional constraints to trade-off the training loss of the network and the time complexity of the model. Our experimental results on CIFAR-10 and ImageNet show that our proposed method confers advantages over several state-of-the-art methods.CIFAR-10 and ImageNet are two commonly used datasets in the field of computer vision for image classification tasks. CIFAR-10 is a smaller dataset suitable for fast experiments and lightweight models; ImageNet is a large-scale dataset suitable for complex models and deep learning research. In particular, compared to the recently proposed BP-NAS, it improves the compression rate by 2.15% and average bit-width by 0.77 bit on ResNet-50, while achieving the same accuracy, and compared to MBF, it improves the classification accuracy by 0.12% on ResNet-20 using the same compression rate and average bit-width conditions. Furthermore, we show on MobileNetV2 that our proposed method achieves 0.23% higher accuracy compared to HAQ.