A Sparsity Based Adaptive Compression Algorithm for Model Compression

With the development of convolutional neural networks, the scale of activation data has increased dramatically, posing significant challenges to model storage and computation. The insight indicates that the sparsity of activations is significantly different across network layers, due to the differences in activation functions. Most existing studies are limited to applying a single compression algorithm to the entire model. This work proposes an adaptive compression method based on sparsity. The optimization strategy includes an adaptive selector that chooses the appropriate compression algorithm based on the sparsity level of each layer, thereby achieving high compression ratios. In comparison to three state-of-the-art compression methods, the proposed algorithm achieves superior compression performance on four mainstream CNN models at different data accuracies.