Initial psychometric evidence of an innovative digital-based learning evaluation instrument for assessing students’ digital citizenship attitudes

This study developed and examined initial psychometric evidence for the Innovative Digital-Based Learning Evaluation (IDBEL), a digital instrument designed to assess students’ digital citizenship attitudes. A quantitative instrument-development design was employed with 661 Grade X and XI students from seven public general upper-secondary schools in Banjarmasin, Indonesia. Before field administration, five experts rated the 25 initial items; Aiken’s V ranged from 0.70 to 0.95 (mean = 0.83), and five below-criterion items were revised. Response patterns, factorability, internal structure, internal consistency, and convergent-validity indicators were examined. The data showed excellent factorability (KMO = 0.963; Bartlett’s χ²(300) = 9,751.236, p < 0.001). The prespecified second-order CFA produced adequate standardized item loadings (0.567–0.776) but mixed and overall unsatisfactory fit: χ²(270) = 1,503.188, χ²/df = 5.567, RMSEA = 0.083, CFI = 0.871, TLI = 0.857, IFI = 0.872, and SRMR = 0.054. Internal consistency was strong (α = 0.953; McDonald’s ω total = 0.958; ω hierarchical = 0.940); domain composite reliability ranged from 0.806 to 0.854. Average variance extracted ranged from 0.456 to 0.540 and reached 0.50 only for Organization, indicating mixed convergent-validity evidence. Strong upper-category concentration, including 97.4% high/very-high responses on Item 12, further limited score discrimination. IDBEL therefore shows promising reliability and preliminary internal-structure evidence, but item refinement and independent validation are required before wider use.