Illusion of Evidence: Accreditation Data and the Case for ACUE Certification

The Association of College and University Educators (ACUE) advertises that its Course in Effective Teaching Practices lowers the rate at which students earn Ds, Fs, or withdraw (DFW rates). A recent large study at Sam Houston State University (SHSU) is presented as independent confirmation of that claim. The same authors applied the same matched-comparison design to a local teaching grant program, and the university’s accreditation report applies it to three more programs. Reading the three documents together, as five programs evaluated the same way, I find that the evidence fits two competing explanations equally well: an effect of ACUE’s curriculum, or the selection of already-effective instructors into selective, paid programs. Across the five programs, the design returned effects ranging from large improvements to a marked worsening, and the local grant program did at least as well as ACUE on DFW rates and on student ratings. The ACUE comparison groups were not balanced on faculty rank, and the reported effect is larger than ACUE’s own meta-analysis predicts. I argue that a study able to tell the two apart would cost most institutions more than it is worth, and that a structured, sustained investment in faculty time is a better use of those resources than a packaged program.Keywords: faculty development, ACUE, propensity score matching, DFW rates, program evaluation, student evaluations of teaching