Revolutionizing neural training with semi-tensor decomposition and adaptive resource allocation

This article presents a training unified Tensor Decomposition model (TUTDM) for neural network optimization. The design goal of this model is to improve the training feasibility of neural networks towards high-saturation-level accuracy. The compression and decompression of hidden-layer functions are used to achieve high accuracy in complex computations. The number of neurons assigned to the training layer is adaptive based on the maximum observed computational complexity. Compression reduces the number of neurons assigned to hidden-layer computations, and decompression increases the number of neurons for highly complex computations. These computational differences are measured based on the saturation-level accuracy rates observed across increasing training iterations. Therefore, the unification of tensors is pursued by using multiple typical neurons to convert the low-to-high saturation deviation. In this case, the training inputs are refined from the conversion point to ensure high accuracy. The tensor decomposition model revises the neuron density from this converting point to improve computational robustness. The proposed TUTDM was validated on the CIFAR-10 and IMDB benchmark datasets, achieving 95.81% accuracy while reducing training time to 12 s, memory consumption to 850 MB, and convergence to 15 epochs, demonstrating efficient and resource-aware neural network optimization.