Beyond Trial and Error: Comparative Analysis of Heuristic Strategies for Optimal Neural Network Architectures

The architecture of a Deep Neural Network (DNN) plays a major role in determining its performance, yet the traditional methods for optimizing these architectures often depend on iterative trial-and-error processes requiring substantial expertise and manual effort. Neural Architecture Search (NAS) has emerged as a rapidly advancing field focused on automating the optimization of hyperparameters and network architectures. This study presents a comparative analysis of three heuristic approaches for NAS: Evolutionary Genetic Algorithms, Reinforcement Learning, and Random Forest Optimization. The efficacy of these methods is evaluated on two widely recognized benchmark classification datasets—MNIST and CREDIT CARD FRAUD—as well as a synthetically generated dataset. A comprehensive evaluation of performance metrics provides insight into the strengths, limitations, and relative effectiveness of each NAS methodology in optimizing neural network architectures for diverse data distributions.