Open Science in Qualitative Evaluation: Considerations and Opportunities

ABSTRACT This article discusses how open science principles—often rooted in quantitative epistemologies—and qualitative evaluation practices can complement each other and explores strategies for evaluators and researchers to consider adopting in their practices. Although the qualitative and quantitative approaches to knowledge creation may be perceived as being in conflict, many practices and values can be viewed as different expressions of the shared goals of rigor and transparency. We describe practices like data, process, and outcome sharing as activities that already align with open science values and encourage evaluators to consider practices like preregistration, registered reports, and replication as possible areas for evaluators to expand into. We also encourage evaluators to contribute to conversations about transparency, community engagement, evaluating effectiveness, and avoiding harm. A flexible, additive approach to evaluation and research projects can allow all parties to draw on each other's strengths for more rigorous, comprehensive, transparent, and community‐centered work. Finally, we suggest a few starting places for evaluators who are interested in incorporating open science practices and researchers who are interested in conducting qualitative evaluations.

Authors

Institutions

Publication Details

Journal
New Directions for Evaluation
Published
2024-12-01
DOI
https://doi.org/10.1002/ev.20627
Citations
2
Primary Topic
Data Analysis and Archiving
Type
article
Field-Weighted Citation Impact
2.26
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

Open Science in Qualitative Evaluation: Considerations and Opportunities

Laurel Standiford Reyes, Crystal N. Steltenpohl, Lindsay Ellis Lee, Mary A. Pei
2 citations
New Directions for Evaluation
Data Analysis and Archiving
2.26
article

Open Science in Qualitative Evaluation: Considerations and Opportunities

Laurel Standiford Reyes, Crystal N. Steltenpohl, Lindsay Ellis Lee, Mary A. Pei
article en
2 citations

Abstract

ABSTRACT This article discusses how open science principles—often rooted in quantitative epistemologies—and qualitative evaluation practices can complement each other and explores strategies for evaluators and researchers to consider adopting in their practices. Although the qualitative and quantitative approaches to knowledge creation may be perceived as being in conflict, many practices and values can be viewed as different expressions of the shared goals of rigor and transparency. We describe practices like data, process, and outcome sharing as activities that already align with open science values and encourage evaluators to consider practices like preregistration, registered reports, and replication as possible areas for evaluators to expand into. We also encourage evaluators to contribute to conversations about transparency, community engagement, evaluating effectiveness, and avoiding harm. A flexible, additive approach to evaluation and research projects can allow all parties to draw on each other's strengths for more rigorous, comprehensive, transparent, and community‐centered work. Finally, we suggest a few starting places for evaluators who are interested in incorporating open science practices and researchers who are interested in conducting qualitative evaluations.

New Directions for EvaluationVol. 2024(184)
University of Southern Indiana (US), Johns Hopkins University (US), Center for Open Science (US), Michigan State University (US)
Openalex Percentile: Top 9%
Data Analysis and Archiving
2.26
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.