Differential misclassification by time: a proposed validation substudy design to account for time trends in bias parameters

BACKGROUND: Data from validation substudies help quantify bias due to misclassification of an analytic variable by estimating classification parameters. Studies with long enrollment or follow-up may be susceptible to changes in classification parameters over time, which would have important implications for validation substudy design and quantitative bias analysis. Herein, we provide guidance for sampling validation data to account for time trends in classification parameters. METHODS: Using a simulated cohort of 10,000 observations, we induced exposure misclassification under three scenarios: absence of a time trend, but expected change in exposure prevalence; linear time trends in exposure misclassification; and logarithmic time trends in exposure misclassification. Validation sampling was conducted at the beginning, middle, and end of follow-up. These validation data were then used to impute positive and negative predictive values for the entire cohort over the study period as a function of time. We compared these imputed values with the true values. RESULTS: We demonstrated that, in the presence of a time trend, purposeful sampling allows for estimation of the changing positive and negative predictive values over the course of the study. Estimation of predictive values was accurate under both the linear and logarithmic scenarios, and was closest to the truth when a large proportion of exposure/outcome strata was sampled for the validation substudy. CONCLUSIONS: When a time trend in classification parameters exists, designs that allow estimation of time-varying predictive values should be used instead of conventional validation study designs that estimate a single summary classification parameter over the study period.