Revision Q-matrix in middle school chemistry: a structural equation modeling approach

The Q-matrix serves as a bridge that links items and attributes, and its accuracy affects the results of cognitive diagnosis. Inaccuracy of the Q-matrix are a common issue frequently encountered in cognitive diagnosis research. This study utilizes the topic "composition and structure of matter" from junior high school chemistry as a representative example, employing Structural Equation Modeling (SEM) to validate the original Q-matrix of the cognitive diagnostic assessment tool. This method prioritizes qualitative analysis results from the perspective of disciplinary connotation, while integrating the signs and significance of regression weights, modification indices (<i>MI</i>), and model fit indices, etc. obtained from SEM, to conducted an iterative process of model revision, parameter estimation, and model evaluation, resulting in a better-fitting revised Q-matrix. By employing the generalized deterministic input, noisy "and" gat (GDINA) model, we conducted a comparative analysis between the Q-matrix derived from the SEM approach and those obtained through the following two approaches: the multiple logistic regression-based method utilizing exhaustive search algorithms (MLR-B-ESA), and the multiple logistic regression-based method utilizing priority attribute algorithm (MLR-B-PAA). The findings show that the absolute fit of the Q-matrix derived through the SEM approach had achieved excellent threshold, although it slightly underperformed compared to the benchmark method in terms of comparative data. It is worth noting that the relative fit of the Q-matrix obtained via the SEM approach was superior to that derived from the comparative methods. This suggests that, as the SEM approach emphasizes qualitative analysis grounded in disciplinary connotation, the Q-matrix revision does not strictly conform to the data information obtained from computation. As a result, this may have a certain quantitative impact on the absolute fit. However, in comparative evaluations of methods, the SEM approach exhibits superior performance.