Performance differences with and without differential item functioning in the post graduate admission test in Saudi Arabia based on gender and ability level

This study aimed to investigate Differential Item Functioning (DIF) based on gender and ability level for Post-Graduate General Aptitude Test (PGAT) items in Saudi Arabia, using classical methods (MH <i>χ</i> ², MH-LOR, BD <i>χ</i> ², and CDR). The study samples consisted of (4,000) students distributed equally between males and females. For overall sample, 56 (54%) out of 104 items showed DIF: with (48%) of them favoring females and (41%) favoring males. For high ability sample, percentage of DIF items decreased across subtests, particularly for verbal sub-test. DIF items favoring females decreased (to 40%) and the ones favoring males increased (to 55%). ANOVA results showed that for overall sample, females outperformed males on total score and verbal ability, while males outperformed females on quantitative and logical abilities, significantly (<i>p</i> < 0.01). When DIF items were removed for overall sample, gender gap was reduced except for verbal ability, favoring females. For high ability sample, differences on total and sub-scores were not statistically significant except for quantitative ability, that favored males (<i>p</i> < 0.01). When DIF items were removed for high ability sample, gender differences were not statistically significant (<i>p</i> > 0.05). Thus, it was recommended to conduct stratified DIF analysis for ability admission tests based on ability area and level, gender and their interaction; and to report DIF size and direction for ability groups based on cut scores.