Revisiting natural selection: evolving dynamic neural networks using genetic algorithms for complex control tasks

Abstract Reinforcement learning (RL) and Genetic Algorithms (GAs) are widely used in decision-making and control tasks, but they often suffer from prolonged training times and inefficiencies. This paper addresses the need for a faster and more precise method to train neural networks in RL tasks, without sacrificing performance. The proposed approach enhances GAs by introducing mechanisms that optimize network architectures dynamically, minimizing unnecessary complexity while maintaining accuracy. The methodology includes a dynamic architecture adaptation technique that trims the neural network to its most compact and effective configuration. A Blending mechanism is introduced to improve the propagation of essential features across network layers, reducing the usage of non-linearity until necessary. An experience replay buffer is integrated to avoid redundant fitness evaluations, significantly reducing computational overhead. Additionally, a novel approach combines back-propagation with GAs for further refinement in supervised or RL tasks, using it as a mutation method to fine-tune the model. Experimental results demonstrate convergence speeds of around several seconds for simple tasks with well-defined rewards, and several minutes for more complex tasks. Training time is reduced by nearly 70%, and the approach provides faster inference speeds due to minimal architecture, making it applicable for mobile and edge devices. The method reduces computation, especially during inference, by over 90% due to the extremely low number of parameters. The performance metrics show comparable results to conventional approaches at the end of training. The proposed method is scalable and resource-efficient, outperforming existing neural network optimization techniques in both simulated environments and real-world applications. The developed framework is publicly available under the MIT license at https://github.com/AhmedBoin/atgen offering an open-source solution for the broader research community.