An Investigation Into Item Calibration in Multidimensional Multistage Testing

Abstract This study builds on prior research on adaptive testing by examining the performance of item calibration methods in the context of multidimensional multistage tests with within‐item multidimensionality. Building on the adaptive module‐level approach, where test‐takers proceed through customized modules based on their initial performance, this research investigates how different calibration methods perform under certain conditions. Specifically, the study evaluates three calibration methods—concurrent calibration, fixed item parameter calibration, and concurrent calibration with multiple panels—within a multidimensional multistage test framework. Using computer simulations, the study assesses ability and item parameter recovery across various conditions, including sample size, correlations among dimensions, and routing stage length. Across 36 simulation conditions, each replicated 10 times, results show that although calibration methods exert minimal influence on item and ability parameter estimates, the correlation among dimensions plays a significant role in both item and ability estimation. Additionally, sample size and routing stage length notably impact the estimation of item discrimination parameters. This study lays the foundation for further research and practical advancements in multidimensional multistage testing, offering a starting point for refining and innovating testing practices.