Auto-grading in Computing Education: Perceptions and Use

Auto-grading technologies have become increasingly prevalent in computing education, driven by the need to handle growing class sizes and provide timely and effective feedback. We conducted a survey of 44 computer science instructors at various institutions in order to gather instructor experience and use of auto-graders, the features instructors value most, and the challenges and limitations faced when using these tools. We specifically asked about factors such as grading strategies and policies, opinions on existing tools, and other automated grading methods they employ. Our results indicated that instructors prefer tools that offer significant customizability and integration capabilities, with functionality and program output-based grading as the most commonly used approaches. They emphasized the need for integrated auto-grading solutions that include robust core features and prioritize extensibility to better align with pedagogical goals and to support instructors in managing the increasing demands of computer science education. Based on these findings, we conclude that existing solutions should be improved to address instructor-reported preferences and diverse educational needs.