When Can Ultra-Short Paired-Comparison Assessment Recover Trait Information? A Simulation and Empirical Illustration

Ultra-short comparative assessments reduce respondent burden, but compression also reduces the information available for score recovery. This article evaluates ultra-short graded paired-comparison (GPC) assessment as a design-contingent information-recovery problem. Study 1 examined 324 Monte Carlo data-generating conditions varying construct saturation, stimulus validity, nuisance variance, pair construction, and test length, with three response formats and multiple recovery methods evaluated within each condition. Across 1,000 replications per condition, design factors dominated average format differences. GPC responses recovered benchmark information under favorable conditions, retained more information than binary paired choices, and deteriorated when target signal was weak, nuisance variance was systematic, or pair construction was contaminated. Reliability could increase while benchmark recovery declined. Study 2 applied this framework to three-pair image-GPC versions of the STAI-6 and SABAS. The STAI implementation showed strong benchmark recovery across constrained ordinal scoring, keyed and component scores, cross-validated prediction, and descriptive multitrait-multimethod checks. The SABAS implementation showed weak recovery across the same diagnostics. The findings support a conservative conclusion: ultra-short GPC assessment can work, but only when the construct, stimuli, pair construction, response format, length, and scoring method jointly preserve enough target-relevant information.