Optimizing Training Hyperparameters for Multilayer Perceptrons in Deep Learning

Deep neural networks require an appropriate selection of hyperparameters for their operation. Among these, training hyperparameters, such as the learning rate, batch size, or the parameters of optimization algorithms, play an important role in the models’ convergence, accuracy, and generalization ability. This paper reviews the most influential training hyperparameters in the context of multilayer perceptrons (MLPs). An analysis of recent studies highlights the approaches used to optimize these hyperparameters and their impacts on model performance. Finally, this paper explores current challenges related to the optimization of training hyperparameters and suggests future research directions.