Enhancing CNN Classification with Lamarckian Memetic Algorithms and Local Search

Optimization is critical for achieving optimal performance in deep neural networks (DNNs). Traditional gradient-based methods, such as stochastic gradient descent (SGD) and ADAM, often face challenges like local minima entrapment and slow convergence rates. This paper explores population-based metaheuristic optimization algorithms for image classification networks. We propose a novel approach integrating a two-stage training technique with population-based optimization algorithms and local search capabilities to enhance solution quality and convergence speed. In the first stage, a global search is conducted using a population-based algorithm to explore the solution space extensively, allowing the network to escape local minima and find promising regions. In the second stage, a local search fine-tunes the solutions from the global search, ensuring improved accuracy and stability. Our experiments show that the proposed method outperforms state-of-the-art gradient-based techniques like ADAM in accuracy and computational efficiency, particularly in scenarios with high computational complexity and numerous trainable parameters. The results suggest that our approach offers a robust and efficient alternative for weight optimization in convolutional neural networks (CNNs). Future work will explore integrating adaptive mechanisms for parameter tuning and extending the method to other types of neural networks and real-time applications.