Open Science Practices Among Evaluators: A Survey of AEA Members

ABSTRACT This study investigates the extent to which evaluators are aware of, understand, and use open science practices in their evaluation work. Members of the American Evaluation Association were surveyed to gather insights on their open science practices, including the sharing of evaluation plans, materials, data, and reports. The study also explored the facilitators and barriers to adopting open science, the alignment of these practices with professional evaluation standards, and the relevance of the replication crisis to evaluation. While some evaluators are sharing evaluation products—primarily reports—many report limited knowledge of and perceived importance of open science. Significant barriers hinder the sharing of evaluation products, such as concerns about confidentiality, contractual and funder restrictions, and fears of reputational harm or criticism. Additionally, evaluators’ understanding of open science may differ from that of established open science practitioners, highlighting opportunities for the evaluation field to further explore the potential benefits of open science practices. The survey, data, and preprint for this project can be found on the Open Science Framework at https://osf.io/r2sh7/ .