Detecting Interviewer Fraud in the 2023 Belgian Health Survey: Methods and Implications

Abstract Background Unreported deviations from standardized interviewer protocols pose a serious threat to the validity and reliability of survey data. Despite established quality control mechanisms - such as the analysis of paradata - anomalous patterns emerged in the 2023-2024 Belgian Health Survey, warranting an in-depth investigation into potential interviewer misconduct. Methods To identify deviations in interviewer behavior, a response deviation score was computed for each interviewer. This metric was derived by comparing the distribution of 20 key survey variables from individual interviewers against the aggregate distribution across all other interviewers. The deviation score, ranging from 0 to 40, was based on the number of statistically significant discrepancies, with higher values indicating greater divergence suggestive of irregularities. Results The median deviation score was 2, and the 90th percentile threshold was 8. A focused analysis was conducted on the 10 interviewers with the highest deviation scores (ranging from 10 to 16). Among them, three interviewers exhibited response patterns indicative of potential data fabrication or procedural violations. Verification efforts, including an examination of physical consent forms, were undertaken to substantiate these findings. A face-to-face inquiry resulted in the full confession of one interviewer, who admitted to fabricating responses. Additionally, evidence suggested that two other interviewers engaged in protocol violations, including skipping questions and manipulating routing mechanisms. Conclusions The data collected by the three interviewers in question were excluded from all subsequent analyses. This case underscores the susceptibility of interviewer-administered surveys to data falsification and highlights the critical need for enhanced fraud detection methodologies. Implementing more sophisticated and proactive quality control mechanisms is essential to safeguarding the integrity of survey research. Key messages • Surveys administered by interviewers are susceptible to interviewer-induced bias and fraudulent practices. • Sophisticated and proactive quality control mechanisms are essential for safeguarding the integrity of survey research.