Most Significant Change: A Technique to Evaluate Successes in RADx-UP C2G Pilot Projects

Introduction The Rapid Acceleration of Diagnostics-Underserved Populations (RADx-UP) Program seeks to ensure that all Americans, especially those from underserved and vulnerable populations, can access COVID-19 testing to reduce COVID-19 related morbidity and mortality. RADx-UP's pilot program, the Community Collaboration Grant (C2G), provides additional support to advance capacity, training, and support for community organizations in underserved communities. Description of Evidence Our team employed a modified Most Significant Change (MSC) approach, a participatory monitoring and evaluation technique, to better understand what C2G awardees felt was the most significant change in the communities they served (Davies & Dart, 2005). These responses were then coded using domains of change, or categories of significant change (Davies & Dart, 2005). Description of Program Activities and Outcomes Our evaluation team interviewed projects eighteen months after they began to understand activities and their outcomes and activity impacts. As of 2024, 70 projects have been funded, and we have interviewed 37. Within a larger interview guide, we asked project team members about the most significant change in their communities from the C2G grant. Rather than following the traditional 10 steps in MSC, we analyzed 5 domains of change: 1) changes in COVID-19 testing access and uptake; 2) changes in COVID-19 vaccination access and uptake; 3) changes in COVID-19 education; 4) changes in community outreach and engagement; and 5) changes in resources and health equity. Leaders most frequently felt their projects’ most significant changes were increased COVID-19 vaccination access and uptake, increased COVID-19 education, and adapting and sharing resources to promote health equity within their communities. Conclusion Projects impacted communities across all 5 domains, which improved COVID-19 health equity in underserved communities. Recommendations Although we adapted the standard MSC method, it was still a useful approach for examining program changes in underserved communities that evaluators may consider implementing. While we did not fully adopt the MSC’s approach, others could complete the traditional ten steps (Davies and Dart, 2005). Davies R, Dart J. The “Most Significant Change” (MSC) Technique A Guide to Its Use. Vol 1.; 2005. Accessed March 1, 2024. https://www.mande.co.uk/wp-content/uploads/2005/MSCGuide.pdf