Parallel Swarm Propagation for Neural Networks

Deep learning has revolutionized various fields but faces challenges such as limitations owing to vanishing gradients and the inability to handle non-differentiable loss functions and layers. Our study proposes Parallel Swarm Propagation for Neural Networks, a novel approach that leverages parallel particle swarm optimization (PSO) for optimizing neural networks. PSO does not require gradient calculations and effectively circumvents the vanishing gradient problem. This enables the exploration of non-differentiable loss functions and layers, potentially leading to more robust and flexible deep-learning models. Additionally, the inherent parallelism of PSO allows for significant performance gains by utilizing the ever-increasing processing power of parallel-computing architectures. Our study investigates the efficacy of parallel PSO in optimizing neural networks, analyzing its performance compared to traditional gradient-based methods, and exploring its potential for unlocking new possibilities in deep learning.