Improving the Measurement of Financial Literacy by Modeling Noncorrect Responses

Abstract: Very brief financial literacy tests are sometimes used in large surveys to reduce respondent burden, but when scored conventionally, they may appear too short to provide reliable or valid measurement. Using data from the Dutch LISS panel ( N = 4,860), this study examines whether more fine-grained modeling of noncorrect responses provides meaningful information about financial literacy. Within an item response theory framework, we compare conventional binary scoring with alternative models that incorporate information from distractors and nonsubstantive response options. Particular attention is given to the nested logit model because it offers category collapsibility and preserves the substantive distinction between correct and noncorrect responses that is central to knowledge testing. Results show that this model yields higher reliability, particularly at low to average ability levels, and produces ability estimates that are more strongly associated with external validity criteria than conventional binary scoring. Our findings suggest that individual differences in financial literacy are reflected not only in whether answers are correct but also in how individuals respond under uncertainty, and that modeling this information can substantially improve measurement in short survey tests.