A Two-stage Self-adaptive Method for Optimal Neural Network Design for Various Time Series Datasets

To deal with time series data prediction, engineers face the problem of designing suitable neural networks. The difficulties lie in three folds. First, many types of neural networks can be chosen as candidate neural networks. However, which one is the most suitable cannot be known in advance. Thus lots of try-and-errors happen. Second, even the type of neural network is determined, the structure of this type of neural network still be unknown. For example, how many layers should we use, how many neurons in each layer should be designed, and so on. This kind of problems cause another round of try-and-errors with limited return. Third, many types of neurons can be mixed in a neural network, for example, LSTM+GRU, TCN+LSTM and so forth. Those combinations are so many that engineers do not know which combination will lead to the best result, thus resulting in another round of unintended try-and-errors. Those above-mentioned phenomena make neural networks difficult to apply for engineers in real application. Thus this paper proposes a two-stage self-adaptive method for neural network design. The first stage will choose the best neural network structures and the second stage optimizes the network weights to further improve the performance of the neural network. Any dataset given to our method will obtain the most suitable neural network with satisfying results. The proposed method is tested on an open dataset and also successfully applied to a real engineering dataset.