Assessing fidelity to Critical Time Intervention (CTI) for homeless-experienced persons

BACKGROUND: It is challenging and time-intensive to assess fidelity to complex evidence-based interventions in the homeless service sector. We describe a multi-component procedure to assess fidelity to Critical Time Intervention (CTI)-an evidence-based, time-limited case management practice for homeless-experienced persons undergoing housing transitions-developed for a large-scale pragmatic trial conducted with homeless-experienced Veterans. METHODS: Using literature review, expert consultation, and pilot testing, we developed a pragmatic, scalable CTI fidelity assessment procedure. We integrated data from: a CTI implementation self-assessment designed to enhance provider practice; 90-minute videoconferences with case managers to collaboratively review charts from ≥2 randomly selected "exemplar cases" assessed for CTI's core components; and field notes from case manager narratives during these videoconferences. At 12- and 18-month timepoints after CTI implementation began with 17 case managers across 15 homeless service agencies, we employed this procedure to assess fidelity to CTI. We used field notes to contextualize differences between self-assessment and expert-rated findings. RESULTS: All case managers self-assessed their CTI practice as well- or ideally-implemented. Expert-rated assessments suggested that all case managers had limited fidelity to at least one of CTI's core components; 4 agencies had inadequate fidelity to the overall practice. Field notes provided explanations for disparate results between the self- and expert-rated assessments, including skill deficits, staff turnover (resulting in limited understanding of CTI's core components), and agency mandates (e.g. for case management visit frequency that were misaligned with CTI). CONCLUSIONS: The effectiveness of CTI is contingent on adherence to its core components. Streamlined approaches to fidelity assessment across EBPs in the homeless service sector is important for pragmatic, large-scale implementation efforts. The integration of self-assessment and expert-rated assessments can be applied to other multifaceted EBPs in the homeless service sector. Supplementing fidelity assessments with field notes may be useful to contextualize disparate findings between self- and expert-rated assessments, and to shape feedback to providers that enhances practice fidelity and improves housing outcomes. TRIAL REGISTRATION: This project was registered with ClinicalTrials.gov as "Implementing and sustaining Critical Time Intervention in case management programs for homeless-experienced Veterans." Trial registration NCT05312229, registered 4/4/2022.