Assessing Accuracy and Weighting Effects in Matched Sample Surveys: Longitudinal Evidence from the Cooperative Election Study

For decades, economists have relied on survey data to track societies and to test models of economic behavior. These survey data were most often collected from random samples of populations, often called probability samples. Recently, nonprobability samples have grown in popularity among researchers using surveys across economics and other fields, with proponents claiming that they can yield accurate measurements at lower cost than probability samples. Despite a growing body of empirical evidence which shows that nonprobability samples are less accurate, some companies and studies have claimed that certain advanced methods of nonprobability sampling can produce results comparable to or even superior to traditional probability sampling techniques. This thesis contributes to the literature by evaluating the accuracy of a large set of nonprobability sample survey data collected by the company YouGov, which claims that its approach to nonprobability sampling, called "sample matching," yields highly and consistently accurate results. The YouGov data were collected for the Cooperative Election Study over 17 years, allowing the assessment of changes in accuracy over time in the measurement of election outcomes and demographics. To distinguish between the effect of sampling matching as opposed to other post-survey adjustments, accuracy is further assessed under different weighting scenarios: before any post-stratification weighting is done (to assess the success of the matched sample on its own), after the application of weights provided by YouGov, and after the application of a new set of weights generated using using optimal methods designed by the American National Election Studies. Results suggest that CES error in measuring election outcomes is on average above 7%, and in some cases as high as 15%, higher than previously thought. Contrary to claims made by providers of nonprobability sample surveys, accuracy has not improved over time.