Karcı fractional artificial neural networks (KarcıFANN): a new artificial neural networks model without learning rate and its problems

The learning rate parameter used in classical artificial neural networks (ANNs) designed with stochastic gradient descent causes problems such as failure to learn, getting stuck in local minima, memorization, and long training times (divergence problem). To address these issues, this paper proposes a novel ANN method that uses a fractional derivative instead of Newton’s derivative. This method is referred to as Karcı fractional ANN (KarcıFANN). In classical ANNs, the weight update is done by assigning the same constant value to the learning rate in each iteration or for a set number of iterations. In contrast, in KarcıFANNs, the weight update process is carried out by calculating the fractional derivative based on the error value in each iteration. Thus, in KarcıFANN, external intervention in the network is minimized compared to that of classical ANNs. KarcıFANN and classical ANN methods were compared for the classification of MNIST and fashion-MNIST datasets. The KarcıFANN method produces successful results for fractional derivative orders between 0.8 and 1.8. The highest accuracy values obtained in the classification of the MNIST dataset were 99.39% for KarcıFANN and 99.43% for classical ANN in the training phase, and 96.76% for KarcıFANN and 96.72% for classical ANN in the validation phase. The highest accuracy values obtained in the classification of the Fashion-MNIST dataset were 98.10% for KarcıFANN and 98.11% for classical ANN in the training phase, and 88.56% for KarcıFANN and 88.54% for classical ANN in the validation phase. The experimental studies show that KarcıFANN is a competitive alternative to classical ANN. KarcıFANN learns faster than classical ANN and eliminates the learning coefficient problem. Additionally, it is seen that the KarcıFANN method is more successful in global modeling. However, achieving the optimal model in KarcıFANN depends on the dataset and hyperparameters.