Plausible Values and Multilevel Models in Large-Scale Assessments

Institutional datasets of large-scale assessments (LSAs) typically contain a set of multiple imputations known as plausible values (PVs) that serve as a proxy for measures of latent proficiency. These are a set of draws from the posterior distribution of latent proficiency that account for measurement error. The PVs are typically drawn from a conditioning model that also includes information from covariates. The advantage of using PVs is that they can be readily used to provide estimates of population quantities of interest at the individual and group levels in secondary analysis of institutional datasets. In this study, the suitability of the PV methodology for secondary multilevel analyses is investigated. It is theoretically shown that consistent estimates for multilevel regression effects are obtained, even when using a single-level conditioning model for the PVs. However, when ignoring the hierarchical structure in the data in constructing PVs, simulation studies showed that the statistical inference is biased and that Type-1 errors, standard errors, and confidence intervals are invalid. The implications for school-level analyses in LSAs are discussed in light of the results.