Improving the Measurement of Students’ Composite Ability Score in Mixed-Format Assessments

Mixed-format assessments, which include both multiple-choice (MC) and constructed-response (CR) items, often produce separate scoring scales, with MC items scored dichotomously and CR items scored polytomously. Conventional methods for estimating composite ability scores, such as weighting or summing, rely on subject matter expertise but overlook the information embedded in MC item scores. While recent progress takes advantage of empirical Bayes analysis for estimating composite ability scores, it may also introduce biases because it relies solely on point estimates without accounting for the variability in unknown ability distributions and other parameters. To address these gaps, this study introduces a practical and easily implementable method, empirical fully Bayesian, that leverages MC item scores to derive empirical priors, leading to more accurate composite score estimates. It has been found that MC scores have effectively captured students’ achievement in the assessment domain, and they could provide valuable information for final scoring. Through empirical analyses of students in Grades 3 to 10 and two additional simulation studies based on real-world data, we demonstrate that this approach enhances composite ability score reliability, reduces reporting biases, and provides a valuable empirical evaluation tool for mixed-format assessments.