An Adaptive Dropout and Parallel Computing Approaches for Accelerating RNN Controller

The efficiency and performance of neural network (NN) controllers present a significant challenge in the rapidly evolving landscape of real-time closed-loop control systems, such as those used in solar inverters. This paper introduces a novel approach that enhances training efficiency by combining adaptive dropout with parallel computing techniques, utilizing the Levenberg-Marquardt (LM) algorithm and Forward Accumulation Through Time (FATT). Unlike traditional dropout methods that apply a fixed dropout rate uniformly across all neurons, Adaptive Dropout dynamically adjusts the dropout rate based on each neuron’s calculated importance and its stage in the training process. This allows for the protection of more critical neurons while regularizing less significant ones, thereby improving convergence speed and enhancing generalization in the neural network controller. To further accelerate the training process, the Adaptive Dropout method is seamlessly integrated into a parallel computing architecture. This architecture employs multiple cores to compute Dynamic Programming (DP) costs and Jacobian matrices for various trajectories simultaneously. This approach not only harnesses the computational power of modern multi-core systems but also ensures efficient processing across all trajectories. The experimental results demonstrate that adaptive dropout with parallel computing provides improvements in training efficiency and overall performance than both no dropout and weight dropout control schemes.