Exploiting Small-Scale Neural Network Models for Dynamic Response Prediction Problems

Neural network techniques have been widely adopted for various tasks, including sequence time-series prediction. Researchers tend to develop large-scale neural network models with multi-level frameworks and extensive numbers of parameters. However, this strategy might not be appropriate for the Dynamic Response Prediction (DRP) problem because large-scale neural network models may tend to overfit the data rather than learn the system’s inherent dynamic property. This paper advocates that neural network models with concise architectures and the ability to make step-by-step moving-forward predictions are better suited for DRP problems. Building on this opinion, the Elman recurrent, long short-term memory, discrete-time state-space, and autoregressive neural networks are adopted in this paper to construct surrogate models for DRP tasks. Numerical and experimental evaluations of these models and other large-scale neural network models are conducted, demonstrating that the four selected types of neural network models, built with limited numbers of parameters, could achieve good performance and minimal overfitting, with overall R-squared values exceeding 90%. It is recommended to utilize these small-scale neural networks or their modifications for DRP problems rather than to construct complicated and large-scale neural networks.