A Generalized Forced-Choice Diagnostic Classification Model Based on Thurstone’s Law of Comparative Judgment in Noncognitive Test: Theory and Application

In the context of single-stimulus (SS) formats, diagnostic classification models (DCMs) have been employed in noncognitive assessments to obtain fine-grained feedback regarding the respondent’s attributes. However, SS format is susceptible to response biases and faking. As an alternative, forced-choice (FC) format has been widely applied in noncognitive assessments, as it can effectively control response biases and fakability that occur in SS formats. Consequently, the incorporation of the FC format into the DCMs in the noncognitive scenario represents an effective means of enhancement. To the best knowledge, only Huang developed a forced-choice diagnostic classification model (FC-DCM) based on simplified modeling. However, there are several important areas where the FC-DCM can be improved. Therefore, this study aims to propose a generalized forced-choice diagnostic classification model (GFC-DCM) for FC blocks within the framework of generalized DCMs, which can not only provide fine-grained diagnostic information about specific attributes but also control response biases and fakability. To validate the performance of the GFC-DCM, a simulation study was conducted in which the sample size, number of statements, FC format, number of attributes, and distribution of attributes were manipulated. The simulation study demonstrated (a) the GFC-DCM achieved high classification accuracy and satisfactory statement parameter estimation accuracy and (b) the number of statements and the FC format exerted a substantial influence on the outcomes. An empirical study demonstrated the GFC-DCM was superior in terms of model convergence and fit, empirical reliability, and diagnostic consistency compared to existing models.