Simulation study: Seeking fairness in a high-stakes placement test

The primary objective of Cognitive Diagnostic Assessment (CDA) lies in problem solving processes to systematically categorize test takers in mastery/non-mastery groups. What is of concern is using some attributes in reading comprehension test item options which are suspected to bias, and is simulating unbiased test items in a reversed engineering attempt. To this end, 4200 PhD candidates who sat for the examination were randomly selected. A Q-matrix was constructed, and raw data were fed into the R-studio software. Two groups of female and male differences were considered for the Differential Item Functioning (DIF) detection based on the Deterministic Inputs, Noisy “and” Gate (DINA) model. Results of the simulation study revealed that in 100 times of item generations, items were non-significant in favor of gender bias. In conclusion, this study was an attempt to determine whether an item identified with DIF is a correct detection or failed to be identified in fine-grained details.