A Self-Optimizing Deep Belief Network With Adaptive-Active Learning: Dynamic Optimization for Neural Network

Deep belief networks (DBNs) are effective deep learning models and widely used to analyze big data characteristics, extract features, and approximate nonlinear systems. However, due to their deep structure and numerous parameters, DBNs suffer from a time-consuming training process generally. To address this problem and improve the training efficiency without reducing accuracy, this article proposes a self-optimizing DBN with adaptive-active learning (SODBN-AAL). In the proposed SODBN-AAL, an adaptive learning algorithm of hyperparameters is designed to ensure a good accuracy. On this basis, an active learning algorithm is developed based on an event-triggered strategy to improve training efficiency by extracting effective data and skipping invalid ones. As a self-optimizing model, SODBN-AAL combines the advantages of both adaptive hyperparameters the event-triggered active learning (ETAL). The convergence analysis of SODBN-AAL is presented as well. Two simulation experiments on 2D function approximation and water quality prediction are conducted to show the advantages of SODBN-AAL. The results show that, compared with the basic DBN model, SODBN-AAL averagely improves learning accuracy by 77.13% and learning efficiency by 84.83%.