Exploring the Influence of Response Time Allocation on Item Revisiting: Implications for Test‐Taking Strategies in Cognitive Diagnostic Assessments

Abstract Computer‐based assessments offer readily available process data for analysis to gain a deeper understanding of the response process. A common response strategy is item revisiting, which can reduce examinees' anxiety and improve their chances of answering questions correctly, and data on item revisiting are recorded automatically in system logs. The approach reported here is to combine two useful and easily accessible types of process data—item response times and item‐revisiting data—with a cognitive diagnostic model to enhance accuracy, identify examinees' level of mastery in specific skills within a particular knowledge domain, and provide personalized diagnostic feedback. The modeling involves two monotonicity hypotheses: (1) examinees who engaged in more revisiting in previous items are more likely to revisit the current item; (2) a longer accumulated response time on previous items results in less remaining time, reducing the likelihood of revisiting the current item. Unlike previous studies in which response time was modeled separately, the focus here is on examinees' revisiting behavior, thus the response time is included in the revisiting modeling as a covariate. This allows an in‐depth investigation of how the accumulated response time influences revisiting behavior, as well as an exploration of the relationship between response strategy (i.e., item revisiting) and time allocation. The Markov‐chain Monte Carlo approach is used for parameter estimation, and its effectiveness is evaluated using two Bayesian evaluation criteria based on posterior samples. Simulation results show that this method is effective for recovering parameters, and an example analysis verifies the the proposed model.