Beyond Functional Limitations: A Multimethodological Approach to Questionnaire Pretesting of New Disability Measures

Abstract Disability measures often collect information on functional limitations but may not fully capture disability status in the working-age population, and seldom collect the type of disability people have. The present research used a multimethodological approach to assess new disability measures for a federal survey. The questions were designed to capture whether respondents have disabilities that impact the kind or amount of paid work they can do, as well as a novel question that collected type of disability. We used multiple methodologies, including cognitive interviews, card-sort/categorization tasks, and web probing. During cognitive interviews, we used retrospective probes to assess the response process and question sensitivity, and a card-sort task, where participants sorted a list of health conditions into similar groups. Additional participants completed a web-probing study, which used a mix of open- and closed-ended probes and were compared to the cognitive interview results. We used thematic coding to compare findings from all three methods, including unique findings that would have been missed if only one method had been used. Each method supported allowing respondents to select multiple types of disabilities, simplification of response categories, and including examples of health conditions that help distinguish categories from one another and vary in level of severity. The cognitive interviews helped refine response category names, while the card-sort and categorization tasks helped develop appropriate category examples. The web-probing study revealed additional question sensitivity not observed in the other methods. We discuss the advantages of a multimethodological approach to survey research.