Measuring Implementation Outcomes Change Over Time Using an Adapted Checklist for Assessing Readiness to Implement (CARI)

In 2017, the Health Resources and Services Administration's HIV/AIDS Bureau funded an Evaluation Center (EC) to assess the rapid implementation of 11 evidence-informed interventions at 25 HIV care and treatment providers across the U.S. The EC conducted an implementation science-based evaluation, including longitudinal assessment of implementation outcomes as defined by Implementation Outcome Framework (IOF) of the Conceptual Model for Implementation Research. The EC adapted a measure originally designed for implementation readiness to capture seven implementation outcomes and administered the measure to site leadership every six months, from intervention launch through the end of the initiative. The adapted measure demonstrated adequate internal consistency within and across time periods. Individual outcomes changed over the course of implementation, with the greatest period of growth during the first six months. Longitudinal relationships between outcomes posited to be most relevant at early, mid- or late-implementation were not evident in these analyses; rather, relationships between the outcomes were significant within time periods. Finally, there were differences in the trajectory of outcomes based on characteristics of the site's larger context. The use of this adapted measure across multiple implementation settings, assessing multiple interventions, is an important step forward in the comparability of implementation outcomes more broadly.