Do Scratchers Fix Their Bugs? Detecting Fixes of Scratch Static Analysis Warnings

Although block-based programming languages like Scratch simplify the creation of programs, learning to program can nevertheless be challenging. Program analysis tools such as linters therefore support learners by providing feedback about potential bugs in their programs. What such tools cannot do, however, is detect whether these bugs are subsequently fixed correctly, even though this is a crucial aspect of learning to program. In this paper, we therefore introduce fix patterns as a way to automatically detect whether instances of bug patterns found by program analysis tools have been correctly fixed, in contrast to the relevant code just having been deleted. Using fix patterns to analyse a dataset of 32,000 publicly shared Scratch projects containing bug patterns reveals that only 2,566 projects received changes and out of these changes not even 8% fix the bug. This implies most Scratchers do not actually fix the bug patterns in their projects. This not only demonstrates that fix patterns are a valuable tool to understand what learners do, but they also hold promise to help encourage learners, and to inform instructors about the progress of their learners.