Signal-Adapted Kernels and Lazy-Evaluation Gaussian Process Regression

Abstract Constant-bandwidth radial basis function (RBF) regression models are popular due to their simplicity, but they exhibit oscillatory patterns that resemble the Runge and Gibbs phenomenon on many real-world datasets. The Gaussian process regression (GPR) can be considered as an extension of RBF regression in that some RBFs can be posed as stationary kernels, but the GPR can also use non-stationary kernels. The GPR scales poorly for large datasets, and existing GPR approximations are either complicated to implement, still require large computational resources, or suffer from discontinuous border issues if an ensemble of local GPRs are used. In this article, we developed a non-stationary kernel that adapts its shape from the regression data, under the existing dimensional expansion kernel framework. We also proposed an observation model for local training sets that continuously diminish the effect of training entries as they move further from the query input, and developed a corresponding lazy-evaluation GPR model that has continuity guarantees for the predictive posterior mean and variance functions. We compared the presence of oscillations and contour-aware smoothing behavior ability of our methods on real-world rainfall and elevation datasets.