Using researcher designed digital games to examine how game-mechanics impact pragmatic development: Some issues at play

This article makes the case for increased consideration towards variable manipulation and research design in studies that use researcher-designed digital games (RDDGs) for pragmatics instruction. Specifically, this article argues for future studies to adopt Mayer's (2019) value-added approach when using RDDGs to examine the influence of game-mechanics on learning outcomes. In the value-added approach, research compares learning between groups that differ according to a single design element. Previous studies that have used RDDGs to examine pragmatic development have examined sets of game-mechanics common to commercial games (e.g., points systems, leveling, hints) in pre-experimental studies (single group, pretest-posttest studies) or (quasi-)experimental studies where two treatments are contrasted without a comparison or control. To illustrate the limitations of these designs, data from Richards (2024) are reexamined according to these designs. These simulated analyses illustrate potential challenges with interpreting findings when comparison or control groups are not used and when treatments differ by multiple instructional features.