What is success? Understanding and addressing evaluation challenges faced by transdisciplinary teams when developing interventions

It has become increasingly common for individuals to collaborate across disciplines and organizations to develop interventions that address complex, real-world social problems. These collaborations often bring together academic researchers and community practitioners, allowing them to leverage a breadth of experience and expertise. However, these collaborations also pose their own challenges, particularly when it comes to defining what “success” looks like and how it should be measured. To better understand these challenges, we first examine a case study of transdisciplinary collaborative teams designing digital media literacy interventions to address social problems related to information illiteracy. Based on this case, we identify three key challenges that may occur in how success is defined throughout the transdisciplinary intervention design process. First, different people hold different definitions of purpose and success. Second, individuals may hold multiple, contextually based definitions of success. And third, the measures that are feasible for assessing success are not always ideal. These mismatches threaten to waste time and ultimately hinder the ability to make claims about the intervention. We then discuss solutions, sharing guidance and activities to help teams create actionable evaluation plans. Specifically, we present the Taxonomy of Outcomes Levels (TOOL) Framework to improve evaluations. TOOL identifies four levels of outcomes relevant to defining and measuring success: (1) fit and feasibility, (2) engagement, (3) knowledge and skills, and (4) community resilience. Finally, we discuss structural supports for evaluation alignment, outlining how funding agencies, journals, and academic or community partners can aid teams in aligning design and evaluation processes. Taken together, this paper provides a pragmatic approach to support transdisciplinary intervention teams working in a diverse range of social problem spaces in better integrating their expertise and values when designing interventions.