Random‐X Prediction Error Under Model Misspecification

ABSTRACT In recent years, advances in computing, particularly the internet, have enabled the collection of Big Data. This has been the key of the current boom of Artificial Intelligence, fed with huge amounts of data, not necessarily of good quality. Processing them with traditional analyses may be unaffordable and new techniques are continuously being developed. Optimal subsampling addresses this challenge by selecting a representative subset that preserves predictive accuracy while reducing computational and labeling costs. When the sampling strategy depends on an assumed model, a critical issue is model misspecification. We develop a theoretical framework for subsampling under misspecification with the goal to minimize a prediction error metric obtained integrating with respect to the marginal distribution of explanatory variables. A key contribution is a theorem that provides an analytical expression for “the best” inclusion probabilities in a Poisson sampling scheme, by connecting sampling theory and optimal experimental design for probability sub‐measures. Theoretical results are validated through simulations, focusing on simple linear regression with explicit solutions. A real data application is also considered.