Psychometric evaluation of a COVID-19 health literacy scale applying Rasch analysis

Abstract Background Assessing health literacy in times of the COVID-19 pandemic is vital, since health literacy has been highlighted as a personal resource for managing health information. The HLS-Covid-Q22 questionnaire was developed to measure self-reported COVID-19 health literacy in adults. It has been used and validated, mainly using confirmatory factor analysis, in some studies with different target groups. However, several comprehensive validation studies are needed to ensure the feasibility of a novel measurement tool. In the current study, Rasch analysis has been chosen, as this analysis transforms the categorical responses of the items of the HLS-Covid-Q22 into linear measures. Methods The psychometric properties of the HLS-Covid-Q22 were evaluated in a sample of 2187 German adults applying Rasch analysis. The data is from the German COVID-HL school survey that was generated by employing an online survey in school principals during the third infection wave from March to April 2021 in four federal states. Applying the Partial Credit Model for polytomous data, the responses to the HLS-Covid-Q22 items were transformed into linear measures, unidimensionality was tested and several indices to assess the quality of the measurement tool were computed. Results Unidimensionality of the questionnaire could be confirmed. The rating scale steps worked as intended. Participants used every rating category. Item fit could be verified. High reliability indices were found with a person separation index of 3.41, person reliability of .92, item separation of 20.08 as well as item reliability of 1.0. Conclusions The HLS-Covid-Q22 appeared to be a reliable and valid questionnaire which has not been validated by applying Rasch analysis before. Even though Rasch analysis is already a well-established validation method in other disciplines, it should be used to evaluate other health literacy questionnaires more often to gain comprehensive insights into the intended functioning of measurement tools. Key messages • The HLS-Covid-Q22 questionnaire appears to be a valid and reliable measurement tool. • Rasch analysis is an appropriate method to evaluate the psychometric properties of likert-type rating-scale questionnaires.