A Comprehensive Multivariate Generalizability Theory-Based Analysis of Responses to the IPIP-NEO-120

The 120-item International Personality Item Pool-NEO questionnaire (IPIP-NEO-120) was recently developed to measure the same Big Five global domain and nested facet constructs included in the 240-item Revised NEO Personality Inventory (NEO PI-R) but more efficiently and at no cost to users. We summarize evidence of reliability and validity for IPIP-NEO-120 scores reported in previous research and further evaluate the psychometric properties of those scores using multivariate generalizability theory (MGT) procedures based on data collected from 447,500 respondents. We used MGT techniques to derive indices of score accuracy for both norm- and criterion-referencing purposes, produce more appropriate indices of accuracy for domain composite scores, estimate correlations between facet scores within each domain corrected for measurement error, and determine value gained when reporting facet in addition to domain scores. Results revealed that indices of score accuracy for domains generally exceeded those for facets, that facet score constructs were more strongly intercorrelated than would otherwise be inferred, and that 28 out of the 30 facet subscales provided added value beyond associated domain scales. We identify facet subscales most in need of improvement and provide code in R to enable readers to apply all demonstrated techniques to their own data.