Low-Rank Compression of CNN Models Based on Efficient Rank Parameter Learning

Tensor decomposition is an important method for compressing convolutional neural network (CNN) models. However, in the decomposition process, it is necessary to configure appropriate rank parameters for each convolutional kernel tensor. Addressing the difficulty in setting ranks, we propose a low-rank optimization algorithm, which defines a cost function based on information entropy. By solving the constructed optimization problems, the low-rank structure and rank parameters of each convolutional kernel tensor can be learned automatically, such that efficiently achieving global scope automatic configuration of rank parameters while ensuring model accuracy. By testing various CNN models on different datasets, the results show that the proposed low-rank optimization algorithm can obtain all rank parameters in a single training process, and the average accuracy loss of the decomposed model does not exceed 1%.