Enhancing Preknowledge Detection in CAT: A Weighted Residual Statistic Via Posterior Predictive Model Checking

In the continuous administration framework of computerized adaptive testing (CAT), item preknowledge poses a significant threat to test fairness and security of the item bank. Traditional person-fit statistics (e.g., l z ) often fail in CAT due to their reliance on asymptotic normality assumptions that are easily violated in short tests. While cumulative sum (CUSUM) methods are better suited for dynamic testing environments, they still face limitations, including insufficient detection power in short tests, the lack of optimization for preknowledge-specific patterns in standardized residual statistics, and an inherent mismatch with the non-continuous nature of preknowledge behaviors. This study proposes a new weighted residual statistic that amplifies abnormal signals by weighting positive residuals and introduces the posterior predictive model checking (PPMC) framework. By simulating examinees’ personalized testing processes to generate posterior distributions, PPMC establishes individualized critical values, providing a precise and adaptive diagnostic standard for CAT environments. Both simulation and empirical studies demonstrate that the weighted residual statistic, integrated with PPMC, effectively optimizes Type I error rates and exhibits higher detection power across various testing conditions compared to traditional statistics. This study provides a sensitive, robust, and practical solution for operational security monitoring in CAT.