Classification Consistency and Accuracy Indices for Simple Structure MIRT Model

Abstract This study investigates the estimation of classification consistency and accuracy indices for composite summed and theta scores within the SS‐MIRT framework, using five popular approaches, including the Lee, Rudner, Guo, Bayesian EAP, and Bayesian MCMC approaches. The procedures are illustrated through analysis of two real datasets and further evaluated via a simulation study under various conditions. Overall, results indicated that all five approaches performed well, producing classification indices estimates that were highly consistent in both magnitude and pattern. However, the results also indicated that factors such as the ability estimator, score metric, and cut score location can significantly influence estimation outcomes. Consequently, these considerations should guide practitioners in selecting the most appropriate estimation approach for their specific assessment context.