Judging Statistical Significance: The Effects of Standard Error Bar Formats and Visual Displays

The present study examined various graphical displays for illustrating statistical significance. These displays were created following Dunlap and May's (1989) recommendation about representing statistical significance in figures with error bars derived from the pooled estimate of error variance. In the first experiment, 15 graduate students were asked to judge whether the relationships depicted in a series of graphs and tables were significant or nonsignificant. Four significance levels were combined factorially with three standard error bar formats, unidirectional opposite direction, unidirectional same direction, and bidirectional, and two displays, line graphs and bar graphs. A tabular display was also presented for comparison. Subject accuracy, response time, and confidence ratings were measured. Results indicated that subject performance was superior with the line drawings and that the unidirectional opposite direction and bidirectional SE bar formats were more effective than the unidirectional same direction SE bar format. The tabular display was least effective. Experiment 2 focused on the comparison of line graphs with cloud displays, where the standard errors of two groups were represented by horizontal bars of color. Two formats of the cloud formation display were compared to three line drawings and the tabular display. Results indicated that cloud displays, unidirectional opposite direction line drawings, and bidirectional line drawings, were equally effective methods of displaying statistical significance. Collectively, these findings highlight the utility of using the pooled standard error to represent statistical significance in graphical displays.