A data driven approach to address missing data in the 1970 British birth cohort

Missing data may induce bias when analysing longitudinal population surveys. We aimed to tackle this problem in the 1970 British Cohort Study (BCS70). We utilised a data-driven approach to address missing data issues in BCS70. Our method consisted of a 3-step process to identify important predictors of non-response from a pool of ~ 20,000 variables from 9 sweeps in 18,037 individuals. We used parametric regression models to identify predictors of non-response that can be used as auxiliary variables in principled methods of missing data handling to restore sample representativeness. Individuals from disadvantaged socio-economic backgrounds, increased number of older siblings, non-response at previous sweeps and ethnic minority background were consistently associated with non-response in BCS70 at both early (ages 5–16) and later sweeps (ages 26–46). Country of birth, parents not being married and higher father’s age at completion of education were additional consistent predictors of non-response only at early sweeps. Moreover, being male, greater number of household moves, low cognitive ability, and non-participation in the UK 1997 elections were additional consistent predictors of non-response only at later sweeps. Using this information, we were able to restore sample representativeness, as we could replicate the original sample distribution of father’s social class and cognitive ability and reduce the bias due to missing data in the relationship between father’s socioeconomic status and mortality. We provide a moderate set of variables that researchers can utilise as auxiliary variables to address missing data issues in BCS70 and restore sample representativeness.