Comparative analysis of stochastic optimization methods for image classification using convolutional neural networks

This paper presents a comparative analysis of several stochastic optimization methods, provides an overview of their advantages and disadvantages, and displays results of comparative experiments, which conducted with simple configurations of convolutional neural network architectures. To perform the experiments, models were trained on the publicly available Canadian Institute For Advanced Research 10 dataset using various stochastic optimization algorithms, and the methods were then examined in terms of convergence speed and the accuracy of the trained model. The results show that the performance of the optimization methods depends on the dataset and the network architecture configuration, good results across different architectures were demonstrated by Adaptive Moment Estimation with Weight Decay Regularization and Stochastic Gradient Descent methods.