Psychometric Validation of the Cognitive Reflection Test on a Russian-Speaking Sample of Students

The article presents the results of the difficulty, internal consistency and reliability testing of different versions of the cognitive reflection test (CRT; S. Frederick, M. Toplak, R. West, K. Stanovich, K. Thomson, D. Oppenheimer). This ipsative test consists of a number of puzzle tasks, the logic of which includes the presence of an impulsive intuitive incorrect answer that first comes to mind. To answer each of the tasks correctly, the respondent must suppress the first intuitive solution and, continuing to think, come to the correct non-obvious solution. Thus, the cognitive reflection test records three scales: cognitive reflection (CRT-r), intuitive errors (CRT-i) and non-intuitive errors (CRT-n). Some questions have a numeric format, and some are verbal (CRT-V). The total sample of the study was 470 students, 22 % ( n = 102) men and 78 % ( n = 368) women aged 17-33, students of different secondary and higher educational institutions of the city of Perm. The analysis of the percentage of correct, intuitively incorrect and incorrect answers revealed different discriminativity of the test tasks. A series of confirmatory factor analyses revealed several single-factor sets of items with acceptable consistency (separately for CRT-r and CRT-i). The reliability of most versions of the CRT task sets is low, but acceptable (the highest values are for the CRT-r = McDonald's ω up to 0.778). The facts of consistency of the numerical and verbal versions of the cognitive reflection test are discussed. The technique can be used to solve scientific and applied problems, both in psychology and other scientific fields.