Online Calibration for Multidimensional CAT With Polytomously Scored Items: A Neural Network–Based Approach

Online calibration is a key technology for calibrating new items in computerized adaptive testing (CAT). As multidimensional polytomous data become popular, online calibration methods applicable to multidimensional CAT with polytomously scored items (P-MCAT) have been proposed. However, the existing methods are mainly based on marginal MLE with an expectation-maximization algorithm (MMLE/EM), making it difficult to accurately estimate parameters in high-dimensional scenarios without sufficient calibration sample size or suitable initial values. To conquer these challenges, a neural network (NN)-based online calibration framework was put forward. The new NN-based methods differ profoundly from the traditional ones in that the parameter estimates of new items are obtained by learning the patterns between input and output data instead of finding solutions to the log-marginal likelihood. Moreover, an alternative solution was proposed for traditional methods to obtain appropriate initial values. Simulation studies were conducted to compare the NN- and MMLE/EM-based methods under various conditions, and further explore the properties of the NN-based methods. Results showed that both the NN-based methods and the alternative solution found their strengths in recovering the item parameters of new items, while the MMLE/EM-based methods struggled to converge when more than three dimensions were involved in the test.