Assessing the psychometric properties of a Comic Book rating Scale to evaluate health information

Background Although graphical modalities have emerged as a potential medium to communicate health information, few studies have examined measurement instruments used to evaluate such modalities. This study examined the use of a Comic Book Rating Scale (CoBoRs) adapted from an existing Mobile Application Rating Scale (MARS) for assessing the quality of a comic book that delivers health information.Aims The aims of this study were to examine the psychometric properties of the CoBoRs by assessing both the reliability and validity of the scale.Methods A cross-sectional survey was administered to a convenience sample (N = 402) of participants obtained from both internet and college student settings. The CoBoRs scale included 10 items across four dimensions (engagement, functionality, aesthetics, and information quality). The psychometric properties of the CoBoRs were assessed by examining internal consistency and item-item correlations; the construct validity of the scale was examined by both exploratory factor analysis (EFA) and confirmatory factor analysis (CFA) using Analysis of Moment Structure (AMOS) software (i.e., structural equation modeling).Results Respondents were mostly White Non-Hispanic (69.4%), female (65.2%), 18–25 years of age (78.6%), and had a high school education or greater (89.1%). There was a medium correlation between CoBoRs constructs (.35 to .70), and the total scale, indicating excellent internal consistency (Cronbach’s alpha=.91). The “Engaging” subscale (Cronbach’s alpha=.75) demonstrated acceptable internal consistency, while informational (Cronbach’s alpha=.82) and Functional (Cronbach’s alpha=.82) subscales demonstrated good internal consistency. However, the Appealing (Cronbach’s alpha=.65) subscale had lower internal consistency. The EFA yielded one factor loading. However, the CFA highlighted that a four-factor model of the CoBoRs provided more robust fit indices than a one-factor model. Additionally, there were high correlations between the CoBoRs and an investigator-developed scale assessing the quality of the comic book through eight statements.Discussion This is the first study to develop and evaluate the psychometric properties of CoBoRs and may be adapted to evaluate other novel knowledge translation products. Future studies may adapt the CoBoRs to evaluate other comic books and health information products.