Development and validation of a mental eHealth literacy scale

This study aimed to develop a new instrument called the Mental eHealth Literacy Scale (MeHLS) to specifically measure mental eHealth literacy and to assess its psychometric properties in the Chinese population of Hong Kong. The MeHLS was developed by a mixed-method study design with five phases. Psychometric properties were evaluated using both classical test theory and item response theory (IRT) approaches. For construct validity, we used exploratory factor analysis (EFA) and confirmatory factor analysis (CFA). We assessed convergent validity through hypothesized correlation analysis and known-group validity via ANOVA. We measured internal consistency using Cronbach's Alpha and test-retest reliability using the intraclass correlation coefficient. Receiver Operating Characteristic (ROC) was used to determine the optimal cut-off point of the MeHLS. A total of 2064 individuals completed the survey. Both EFA and CFA demonstrated a three-factor structure of the MeHLS, aligning with the design. The IRT analysis showed the discrimination value ranged between 1.580 and 3.296, with no items having disordered thresholds. Correlation analysis revealed that MeHLS was significantly associated with all concept-related instruments, indicating good convergent validity. ANOVA results demonstrated that MeHLS can significantly differentiate individuals in all risk groups. The MeHLS showed good reliability, with a Cronbach's Alpha of 0.95 and an Intraclass Correlation Coefficient of 0.75. ROC analysis confirmed a cut-off score of 67 to differentiate individuals with high and low MeHL. The MeHLS is a multidimensional instrument that shows good psychometric properties for measuring web-based mental health literacy. • This is the first scale developed to specifically assess mental eHealth literacy worldwide. • Findings confirm the scale has sound psychometric properties, using both classical test theory and item response theory methods. • The estimated cut-off points support the new scale as a valid screening tool to differentiate individuals with low and high mental eHealth literacy.