Fidelity Assessment Tool for a Dementia Carers’ Group-Psychotherapy Intervention

Context: The systematic evaluation of a practitioner’s adherence to and competence in delivering psychotherapeutic interventions can be complex. This study describes the development of a fidelity assessment tool for the Reitman Centre CARERS Program (RCCP), a carer group-psychotherapy intervention with multiple didactic and clinical components. The tool’s value in informing psychotherapy training and best practices for practitioners from diverse professional settings is examined. Methods: The RCCP Fidelity Assessment Tool (RCCP-FAT) was developed following an iterative process of item writing and checking. Seven components of the RCCP—Group Structure, Dementia Education, Problem-Solving Techniques, Therapeutic Simulation, Vertical Cohesion, Horizontal Cohesion, and Overall Global Rating—were assessed, with three to eight items, and a “global score” assigned to each. Fifteen trained raters were paired up to rate 36 RCCP sessions using the RCCP-FAT. Rater agreement, correlation between itemized and global scores, and correlation between global ratings and RCCP participants’ satisfaction were calculated. Results: A total of 1188 RCCP-FAT items were rated by each of the two rater groups. Rater agreement was calculated to be 54.3% (κ = 0.32; 95% CI, 0.02681–0.3729). A positive correlation was found between the itemized and global scoring for four RCCP components evaluated (R = 0.833 to 0.929; p < 0.01). The global score and the participants’ satisfaction with “Simulation” was also positively correlated (R = 0.626, p < 0.01). Conclusions: The study provided evidence for fair rater agreement for all RCCP-FAT assessment items. More importantly, the process of developing the tool systematically crystallized the clinical elements of the RCCP and helped to standardize the training methods by creating a framework for providing feedback to learners that matches the items on the RCCP-FAT. The use of the RCCP-FAT to guide the training and mentoring of incoming group leaders is essential in the scaling and dissemination of a complex training method like the RCCP to ensure fidelity to the original evidence-based intervention.