Adaptive Sparse Trajectory-Unified method for Kolmogorov–Arnold Networks Training

Traditional Multilayer Perceptron (MLP) lack interpretability when dealing with complex nonlinear problems, and Kolmogorov-Arnold Networks (KAN) provides a new solution to this problem through learnable activation functions. Nevertheless, KAN has issues with low training efficiency and high-order networks. In order to overcome significant obstacles in KAN training, this work suggests a novel technique termed Adaptive Sparse Trajectory-Unified (ASTJU). The innovation of the ASTJU method include: first, in order to achieve a more compact and efficient architecture without compromising expression ability, an Adaptive Sparse mechanism is proposed; second, the KAN training problem is transformed into a stable equilibrium point (SEP) by building a business, which makes the training process more systematic and efficient; finally, the pseudo-transient continuation (PTC) technology is used to speed up convergence and locate the local optimal solution. Experimental results show that on several datasets, such as the Feynman dataset, MNIST, and CIFAR-10, the ASTJU approach performs better than the conventional optimization method. It performs remarkably well in terms of model convergence speed and parameter utilization efficiency, in addition to greatly increasing test accuracy. In addition to offering a novel approach to KAN network training, this research offers fresh approaches to the interpretability and efficiency problems in deep learning.