Validation of the Observational Assessment Tool for Tailoring (OATT)

Individually tailored interventions can address the myriad multi-level determinants of chronic health conditions. Limited measurement modalities to quantify tailoring disallow examining "active ingredient" effects on outcomes and implementation fidelity. The objective of this study is to develop and validate the Observational Assessment Tool for Tailoring (OATT) for behavioral prevention interventions. We developed the OATT and coded n = 172 videorecorded sessions from two trials of the Family Check-Up® 4 Health (FCU4Health), an individually tailored prevention and management program for behavioral health and obesogenic behaviors with English and Spanish-speaking participants. The sample was culturally diverse (> 65% Hispanic/Latino). Confirmatory factor analysis (CFA) tested the two-factor model. McDonald's Omega estimated internal consistency. Discriminant and predictive validity tests were conducted with FCU4Health fidelity, engagement, and health behavior outcomes, informed by the Implementation Cascade Model. CFA confirmed a two-factor structure for both trials (i.e., RMSEA ≤ 0.06, CFI and TLI of ≥ 0.95, SRMR < 0.08 chi-square p ≥ 0.05). Reliability and inter-rater reliability were good (ICC > 0.77) for both trials and English and Spanish videos. The OATT was not correlated (p > 0.05) with discriminant validity variables. Path analysis for predictive validity indicated that fidelity to the Individualized Treatment Planning factor directly predicts improvements in participant engagement (B = 0.16, p = 0.01, 95% CI [0.03-0.29]), which directly predicts improvements in parent health behaviors 12 months post-baseline (B = 0.18, p = 0.01, 95% CI [0.02-0.34]). The development of the OATT is a critical step to measure and guide tailored intervention development, implementation, and evaluation. Future studies are needed to replicate predictive validity findings and test the OATT factor structure with larger samples and different prevention initiatives.