Assessing information problem solving through a simulation game: Validation evidence from a municipal enforcement official context

Assessing complex vocational skills such as information problem solving is challenging, as it requires balancing authentic, construct-relevant tasks with standardized assessment conditions to ensure fair and meaningful results. This study responds to this challenge by addressing the gap in empirical validation of simulation games for summative assessment by validating <i>Crossroads</i> – a simulation game designed to measure information problem solving among municipal enforcement officials responsible for maintaining public safety. Grounded in the information problem solving model, the assessment focused on three of the model's constituent skills: scanning, processing, and organizing and presenting information. Performance on these skills was operationalized through six subscales. Using a known-groups design, the performance of first-year students, final-year students, and practicing municipal enforcement officials was examined. Participants completed a domain-specific information problem solving task in <i>Crossroads</i>: planning a surveillance route. A confirmatory factor analysis was conducted to investigate the underlying latent structure of the performance data. The results supported a one-factor model, suggesting that performance across subscales reflects a single underlying construct. Significant performance differences were observed between final-year students and both first-year students and practicing officials. Participants evaluated <i>Crossroads</i> positively in terms of usability and face validity. The findings provide evidence supporting the use of simulation games for summative assessment, demonstrating that Crossroads can elicit and differentiate levels of information problem solving in an assessment balancing authentic task characteristics with standardized assessment criteria. Several limitations highlight directions for future research, including expanding construct coverage, eliminating prior knowledge confounds, and balancing game autonomy with assessment structure.