Robust Variational Gaussian Process Regression for Count Data with the Trimmed Marginal Likelihood

Abstract Gaussian process (GP) regression is often the first choice for prediction with well-calibrated probabilities. However, since GP regression can be very sensitive to outliers, various methods have been proposed to improve robustness, for example, by replacing the Gaussian likelihood with a Student-t distribution. Unfortunately, for predicting count data, like the number of hospitalized patients, such methods are not appropriate. As a solution, we propose to use a latent GP regression model with a negative binomial likelihood that is trained by optimizing a trimmed variational lower bound, where the trimming ratio corresponds to an upper-bound on the number of outliers. We formulate the optimization problem using the sparse variational inference framework, which allows us to derive an efficient training algorithm with provable robustness to outliers. Furthermore, since an unnecessary conservative estimate of the outlier upper-bound can hurt statistical efficiency, we also propose an effective refinement method for tuning the trimming ratio in an automated fashion. Experiments on synthetic, and various real datasets confirm that optimizing the trimmed variational lower bound, rather than the standard variational lower bound, leads to improved predictive uncertainties that are almost unaffected by outliers in the training data. We also generalize and compare our approach to the Bayesian data re-weighting framework and the $$\gamma $$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mi>γ</mml:mi> </mml:math> -divergence for count data, showing experimentally that the latter is inferior to our proposed methodology.