Low-Rank CP Tensor Decomposition for Lightweight Image Super-Resolution

Deep convolutional neural networks have substantially advanced single-image super-resolution, or SR, but their high parameter counts and computational redundancy hinder deployment on resource-constrained edge devices. Most existing compression methods apply pruning or low-rank truncation as a post hoc step after training. This “train-then-compress” pipeline is reported in the compression literature to disrupt the spectral coherence of the learned feature manifold and to require costly fine-tuning to restore fidelity. We study an alternative, training-time approach that re-parameterizes the convolutional kernel with CANDECOMP/PARAFAC (CP) decomposition into three low-rank sublayers, optimizing directly on a rank-constrained manifold rather than compressing after convergence. We interpret the forward pass as a heuristic realization, rather than a formal equivalence, of a proximal gradient-descent–ADMM update for a tensor nuclear-norm-regularized problem, which motivates the design and its hyperparameters without invoking expensive singular value decompositions. An analysis of parameter complexity and gradient flow justifies the choice of CP over Tucker and tensor-train decompositions. Experiments on DIV2K, Kodak24, BSD100, and Set12 show that the CP module yields a 0.11 M-parameter lightweight model at parity with a same-budget reduced residual baseline (at most 0.15 dB difference on every test set), sacrificing roughly half a decibel relative to a 5.2× larger residual baseline. Applied to a large-capacity SR backbone, the same module removes 76.7% of the parameters, from 13.28 M to 3.09 M, at a PSNR cost of only 0.11–0.20 dB. These results indicate that training-time CP decomposition is a simple and parameter-efficient route to compact SR models in the extreme-compression regime.