Investigating Differential Item Functioning: A Hybrid Approach Combining Clustering and Statistical Testing

Throughout the social sciences, comparisons between groups on various latent traits are frequently made. To conduct these comparisons objectively, the absence of differential item functioning (DIF) is vital. Despite the pressing need for methods to identify DIF-free items, there is no consensus regarding the best methods to test for DIF. One reason for this may be the fact that item difficulty is not identified from observations. To address this, DIF tests utilizing the pairwise differences between item parameters were proposed for the Rasch, partial credit, and 2-parameter logistic models. Based on these pairwise item parameter differences, clusters of similarly functioning items are formed. While these approaches are promising conceptually and are based on empirically identifiable properties of items, two issues are as of yet unresolved. Specifically, the cluster membership of items may be unclear when using a statistical test to identify item clusters. Approaches using k-means clustering have been proposed to remedy this, but these may create too many clusters of items, potentially leading users to unnecessarily discard DIF-free items and complicating the research into sources of DIF. The current paper thus aims to combine both the statistical testing and k-means approaches to obtain clearly defined clusters without unnecessarily discarding DIF-free items. Performance of the proposed approach is assessed for the Rasch model using a simulation study and compared to the performance of the original k-means based approach. In addition, special cases of the proposed approach are discussed and practical recommendations for applied researchers are made.