Indentation and reading time: a randomized control trial on the differences between generated indented and non-indented if-statements

Abstract Indentation is an old technique that emphasizes elements in source code using white spaces or tabs. But while this technique has been taught and applied for decades, evidence for its effectiveness is weak: up to 2022 relatively few experiments can be found – the present authors are only aware of one single experiment that revealed an effect of indentation and reported the effect size, but even in that experiment the effect of indentation was found to be weak. The situation changed recently, where an experiment was published that suddenly revealed a strong and large effect of indentation in control flows on reaction time. However, although the experiment provided an initial indication of a possible cause for the difference between indented and non-indented code (the length of the code that can be skipped in indented code), the evidence for this indicator was rather weak. The present paper presents a formal model on the differences in reading times (measured in terms of reaction times) between indented and non-indented code. Based on that model a controlled experiment on generated tasks was designed and executed on 27 participants (undergraduate students, PhD students, professionals). The experiment (again) confirms a strong (p &lt; .001) and large ( $$\eta _{p}^{2}$$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:msubsup><mml:mi>η</mml:mi><mml:mrow><mml:mi>p</mml:mi></mml:mrow><mml:mn>2</mml:mn></mml:msubsup></mml:math> = .198, $$\frac{M_{Non-Indented}}{M_{Indented}}$$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:mfrac><mml:msub><mml:mi>M</mml:mi><mml:mrow><mml:mi>N</mml:mi><mml:mi>o</mml:mi><mml:mi>n</mml:mi><mml:mo>-</mml:mo><mml:mi>I</mml:mi><mml:mi>n</mml:mi><mml:mi>d</mml:mi><mml:mi>e</mml:mi><mml:mi>n</mml:mi><mml:mi>t</mml:mi><mml:mi>e</mml:mi><mml:mi>d</mml:mi></mml:mrow></mml:msub><mml:msub><mml:mi>M</mml:mi><mml:mrow><mml:mi>Indented</mml:mi></mml:mrow></mml:msub></mml:mfrac></mml:math> = 2.13) effect of indentation. Furthermore, it confirms that the larger the skippable code is, the larger is the difference between reading time of indented and non-indented code (p = .001, $$\eta _{p}^{2}$$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:msubsup><mml:mi>η</mml:mi><mml:mrow><mml:mi>p</mml:mi></mml:mrow><mml:mn>2</mml:mn></mml:msubsup></mml:math> = .072). I.e., the experiment goes beyond the point “ indented vs non-indented code ” and explains the difference by revealing a factor that controls this difference. However, although the previous statements holds true for the whole sample in the experiment, this effect could only be shown for a subset of individual participants.