Exploring student estimates of astronomical scales: Impact of question formulation and visualization

Estimating astronomical scales requires multiple complex mental processes, such as spatial thinking and interpreting large numbers. As such, it is a nontrivial question how these estimates can be most efficiently assessed. There is reason to believe that results from previous studies probing astronomical scale estimates are possibly susceptible to effects inherent in the questioning methods used. In this study, an interactive online survey was constructed and administered to 201 students in their last year of high school. To probe their estimates of spatial scales in the Solar neighborhood, we formulated five questions, two of those probing estimates on the relative sizes of astronomical bodies and three on the relative distances between those bodies. These questions were formulated in two different ways, and the effect of these formulations was studied. In one formulation, students were asked to numerically compare the magnitude of two sizes or distances, while in the other, these estimates were made in terms of the travel time of an imaginary spacecraft. After every answer, students were confronted with a customized visualization, which they could either agree with and move on to the next question or disagree and reconsider their previous answer until they agreed. Studying the effect of these visualizations on the students’ answers was another objective of this work. Most students had difficulties estimating both the relative sizes of the considered celestial bodies and the distances between them. The range of estimates covered many orders of magnitude for all questions, and for the distance-related questions, there was a clear trend of underestimation. We found a significant impact of question formulation on the magnitudes of student estimates. However, we found no indication that one question formulation led to more reliable results than the other. The effect of the visualizations was smaller than anticipated but noticeably larger for the size-related questions than for the distance-related questions. Self-assessments of certainty were made by the students after every answer, and those were found not to correlate with the accuracy of the answers. The implications of these findings are discussed.