Why Kolmolgorov-Arnold Networks (KAN) Work So Well: A Qualitative Explanation

In the usual deep neural network, weights are adjusted during training, but the activation function remains the same. Lately, it was experimentally shown that if, instead of using the same activation function always, we train the activation functions as well, we get a much better results -- i.e., for the networks with the same number of parameters, we get a much better accuracy. Such networks are called Kolmogorov-Arnold networks. In this paper, we provide a general explanation of why these new networks work so well.