Development of a multiple-choice test to assess critical thinking in climate change

Recognizing the complex environmental challenges that humanity faces, there is a growing need to develop critical thinking skills specifically oriented toward understanding and addressing climate change. To facilitate the assessment of these skills, a critical thinking in climate change () test was developed. In the first step, 16 semistructured interviews were conducted with high school and university students. Drawing on insights from these interviews alongside previous research, a first version of the test comprising 41 items was administered to a sample of <a:math xmlns:a="http://www.w3.org/1998/Math/MathML" display="inline"> <a:mrow> <a:mi mathvariant="italic">N</a:mi> <a:mo>=</a:mo> <a:mn>2</a:mn> <a:mn>8</a:mn> <a:mn>9</a:mn> </a:mrow> </a:math> high school and university students from Austria, Belgium, and Germany. The test development process and subsequent evaluation were performed using Rasch modeling to determine test functioning. Through an iterative process that included three rounds of testing involving a total number of <d:math xmlns:d="http://www.w3.org/1998/Math/MathML" display="inline"> <d:mrow> <d:mi mathvariant="italic">N</d:mi> <d:mo>=</d:mo> <d:mn>8</d:mn> <d:mn>0</d:mn> <d:mn>9</d:mn> </d:mrow> </d:math> students and analyses, the test items were revised. The final version of the test consists of 24 items. Results from Rasch analysis suggest that the test has high item reliability (0.96) and person reliability (0.80), making it a useful tool for assessing critical thinking skills among high school graduates and first-semester university students.