Robust Jackknife Model Averaging

In the age of big data, model averaging has been proved to be a powerful tool for data analysis, which helps to mitigate bias and reduce overfitting that can arise from relying on a single model.However, outliers in large-scale datasets like image recognition and fraud detection can severely degrade traditional model averaging built on least squares or maximum likelihood.To address this challenge, we propose a robust jackknife model averaging (RJMA) approach, where the weights are selected by minimizing a leave-one-out cross-validation criterion.This framework is adaptable to situations where the dimensions of candidate models increase with the sample size.We establish the asymptotic optimality of the RJMA estimator, demonstrating its ability to minimize out-of-sample final prediction errors.We also present the consistency of the proposed weight estimator to the theoretically optimal weight vector.Furthermore, in the scenario where one or more correct models are present in the candidate model set, we show that RJMA assigns all weights to the correct models, leading to a consistent model averaging estimator.Additionally, we derive the influence function of the RJMA estimator and introduce the empirical prediction influence function to quantitatively evaluate its robustness.To illustrate the efficacy of the proposed methodology, we conduct numerical studies including Monte Carlo simulations