How to communicate evidence to teachers: Comparing the effects of verbal and visual effect size representation [Registered Report - Stage I]

Teachers are encouraged to consider relevant evidence from educational research to improve the quality of their instruction. This approach requires them to have a reasonably accurate idea of the effectiveness of educational interventions. Currently prevalent ways of communicating effect sizes, however, often lead to imprecise effect magnitude estimates. We aim to investigate how teachers estimate and perceive verbally and visually communicated effect sizes. For the pilot study, N = 20 English-speaking teachers were recruited on Prolific, and for the main study, N = 120 will be recruited. We employ a randomized 4 × 6 × 2 between–within-within-person experiment with the outcomes of interest being teachers' effect magnitude estimates and perceived relevance, informativeness, and difficulty. Across the conditions, teachers' effect magnitude estimates correlated strongly ( τ = 0.69) with true effect sizes and personified relevance correlated moderately ( τ = 0.41) with objective relevance. However, teachers were less accurate in the verbal than in the visual condition (Cohen's d = −0.52, 95 % CI = [−0.81, −0.23]). Conversely, they perceived verbally communicated Cohen's U 3 expressions as more informative (r rb = −0.27, 95 % CI = [−0.40, −0.13]), and less difficult (r rb = 0.31, 95 % CI = [0.17, 0.44]) to understand than visually communicated effect sizes. Communicating effect sizes can work verbally and visually. • Communicating effect sizes can work verbally and visually. • Teachers are more accurate when estimating visually communicated effect sizes. • Teachers perceive verbally expressed effect sizes to be more informative. • They can successfully distinguish between small and large effect sizes.