Repeated Attempt Models for Nonresponse Bias: Small Sample Performance, Optimization of Contact Attempts, and an Application to Deer Hunters

Abstract Nonresponse bias is a pervasive issue in surveys of people, affecting data quality across the social sciences. Such bias arises when the propensity to respond to a survey varies with the outcome variable, resulting in data that are missing not at random (MNAR). Repeated attempt models (RAMs) leverage information in nonresponses by survey subjects to estimate an outcome variable when data are MNAR. Here, we discuss parallels between RAMs and ecological occupancy models, which also use repeat surveys to account for imperfect detection of animal species during surveys. We describe a few likelihoods for RAMs, and we use simulations to evaluate the small sample performance of a RAM with a binary outcome. We also identify the optimal number of repeat surveys to complete, given different probabilities of response. Finally, we apply a RAM to surveys of deer hunters in Arizona, USA. Simulations indicated that RAMs generated unbiased estimates of a binary outcome for a variety of parameter values, but performance broke down when response probabilities or sample sizes were too low. We found that the optimal number of repeat surveys varied from about 4 to 20, depending on parameter values, but RAMs are likely inappropriate when data are missing at random. Our analysis of deer hunter data using a RAM produced plausible estimates of deer harvest. Relative to the RAM, a propensity model over-estimated deer harvest by 36–129 percent. A RAM could be extended to address unmodeled variation in response probabilities, inaccurate responses, and changes in outcome variables through time. We suggest that RAMs could be broadly useful for many natural resource and social science applications.