The Impact of Item Bank Transition Rules on Student Ability Estimates and Achievement Level Classifications

This paper describes a hybrid interim-summative computerized adaptive assessment design administered across three academic terms (fall, winter, and spring).Each test event had two phases that either stayed on-grade or moved off-grade dynamically.Using three different transition rule conditions, this design was compared to an on-grade hybrid interimsummative computerized adaptive assessment design using a series of simulations for a Grade 4 and Grade 6 mathematics assessment.A 500-and 800-item bank in each grade was simulated with a normal distribution of items that ranged across achievement levels.Simulees were randomly drawn from a normal distribution.Simulees whose ability estimates met the transition rule requirements were routed to off-grade item banks and test blueprints dynamically.During Phase 1, the algorithm was programmed to administer only on-grade items.At the end of Phase 1, a transition rule was used to determine if the student should be routed to off-grade items and blueprints for Phase 2. If the student ability estimate did not meet the transition rule requirements, the adaptive test continued to administer ongrade items.If the ability estimate met the transition rule requirements, the student was routed to item banks and blueprints at the adjacent lower or higher grade, as appropriate.The results were evaluated based on measurement precision (i.e., RMSE), measurement accuracy (i.e., bias), item exposure rates, and classification accuracy.Results indicated that while more students were routed off-grade when more lenient routing rules were used, most