Various-Neighbor Neural Network Emulator for Computer Models with Large Spatial Output

Abstract Computer models (simulators) are frequently used to simulate the behavior of a complex system in various scientific fields because conducting real experiments is expensive, dangerous, or impossible in many problems. However, computer models themselves are often expensive to run and the number of runs one can obtain is quite limited in a typical computational environment. Due to this limitation, it is desirable to build an inexpensive statistical surrogate model to emulate the behavior of computer models. Building an emulator for high-dimensional outputs with the existing standard method, the Gaussian process model, can be computationally infeasible because it has a cubic computational complexity that scales with the total number of runs as well as the dimensionality of the output. In this work, we construct a computationally scalable and flexible emulator based on a deep neural network (DNN) with feedforward multilayer perceptrons (MLP). High-dimensional outputs and limited runs can pose considerable challenges for DNNs to learn a complex computer model’s behavior. To overcome this challenge, we take advantage of the neighborhood structure in both parameter space and the model output space to engineer features for better training of DNN. By utilizing a data augmentation technique to generate various neighborhood configurations, our feature engineering improves the predictive performance in a sparsely sampled area, which is a typical weakness in most nearest neighborhood-based approaches. We apply our approach to two geophysical models, UVic ESCM and the PSU3D-ICE models, to demonstrate that our method results in a better emulation performance than an existing state-of-the-art emulation method. Supplementary materials accompanying this paper appear on-line.