Tuning deep learning parameters using bio-inspired algorithms

There are several domains in which deep learning techniques are successfully applied. The great success of deep learning depends on the great performance of models. Learning rate, choice of activation function, network architectures and so forth, all of these are deep learning hyper-parameters that had an important role in improving the performance of models, but there are no rules for choosing the right value of each hyper-parameter. In this paper, we will use some bio-inspired algorithms like genetic algorithm, particle swarm optimization algorithm, black widow optimization algorithm, and grey wolf optimization algorithm to optimize the CNN and LSTM models hyper-parameters. We have used two data-set, the first is the MNIST data-set which concerned images and the second is the 20 newsgroups text data-set. We have illustrated the results of the tests affected by the bio-inspired algorithms and both random and grid searches. The results show that bio-inspired algorithms give better accuracy results that can be the way to tune deep learning hyper-parameters.