How internal noise impacts simplified deep neural network during training

In this paper, we analyse the impact of additive and multiplicative uncorrelated white Gaussian noise on a simplified deep neural network during training. For this purpose, we train artificial neural network to recognize the set of handwritten digits (MNIST database). The network consists of three layers (one input, output and hidden layer), where hidden layer has 20 neurons with sigmoid activation function. The internal noise in artificial neural networks can arise in the case of analog (hardware) neural networks, which artificial neurons and their connectivity are based on physical principles. Therefore, it is of particular interest to study how internal noise impacts the final performance. Here we introduce the noise into the neurons of hidden layer. We show, that for some cases additive and multiplicative noise can increase the training speed.