A Software Framework Based on JUnit for Automated Software Testing in Computer Science Courses

This innovative practice full paper describes a new software framework based on JU nit to test student work. Automated testing is an important capability when teaching software development at the college level. Ideally, a testing system will allow the instructor to efficiently create a thorough set of tests. Also, the software should facilitate grading tasks and produce informative reports that can be distributed to the students in a timely fashion. For Java development, the well-known JUnit framework enables a test suite to be applied to a student's submission. The mutools library presented here extends the JUnit framework in novel ways to accelerate the instructor's task of creating test suites. This new framework allows the instructor to augment tests with directives to control scoring and reporting. The four main capabilities of the software include: 1) An assert statement that does not terminate the test when it has failed. Instead, statistics are maintained regarding the success or failure of each assert statement. 2) Tests that can be configured to award partial credit. This can be useful in situations where the instructor deems it appropriate to award students some credit even in the presence of incorrect asserts. 3) Tests that can be grouped into categories that match a particular rubric item. Java annotations are placed on the test suite to define these categories. For example, @TestCategory(name= “remove”, points=10.0) specifies that 10 points will be awarded for successful implementation of all tests related to removing an item from the collection class. 4) Testing reports that contain varying levels of information. With minimal details, the testing report shows each testing category with the following information: assert statistics, whether the test timed out or had abnormal termination. This software has been used for many semesters and has been found to increase the speed at which the instructor can develop test suites for grading. The framework is available on github.