Development and initial validation of a situational judgment test for the measurement of actively open-minded thinking

Abstract Existing measures of Actively Open-Minded Thinking (AOT) primarily assess the acceptance of rational thinking norms and standards, rather than actual thinking and resulting behavior. These scales can be susceptible to impression management, often yield inflated scores, and may not accurately capture how individuals think in real-life contexts. To address these limitations, we developed and validated a novel Situational Judgment Test for Actively Open-Minded Thinking (AOT-SJT), designed to assess behavioral tendencies related to AOT in realistic scenarios. AOT is conceptualized as the disposition to consider alternative viewpoints, seek disconfirming evidence, and revise beliefs in light of new information. Across 4 studies, we constructed and refined the AOT-SJT using scenarios that simulate everyday decision-making. In Study 1, we tested initial items among Croatian participants, resulting in a 13-item measure with solid psychometric properties. Study 2 confirmed the test’s convergent validity with cognitive and personality constructs and its predictive power for different forms of rational thinking. In Study 3, new items were introduced to enhance construct coverage, particularly around evidence search direction. Study 4 extended validation to an English-speaking sample, supporting cross-linguistic applicability, although effect sizes related to convergent validity were somewhat lower than before. Findings across studies show that the AOT-SJT aligns with theoretical expectations, demonstrates solid convergent validity with existing AOT scales, and effectively distinguishes levels of open-mindedness. By measuring behavioral intentions rather than standards acceptance, the AOT-SJT offers an externally valid assessment of AOT.