Variability, Reliability, and Multilevel Determinants of Care Quality for Substance Use Disorders

OBJECTIVE: Two quality measures, initiation and engagement in substance use disorder (SUD) treatment (IET), have been adopted by stakeholders. Further evidence is needed to inform optimal use of these measures to support quality improvement efforts. The objective of this study was to examine variability, reliability, and multilevel correlates of IET. METHODS: Retrospective cohort study of members of a large integrated health system in the Pacific Northwest. Descriptive statistics were used to characterize variation in quality. Mixed-effects models with random intercepts were applied to quantify variability and reliability at different health system levels. We examined associations between quality measures and a wide set of patient-, provider-, clinic-, and area-level characteristics. We examined 32,960 care episodes subject to IET quality criteria between 2017 and 2022. RESULTS: Overall, 30.0% and 7.8% of episodes met the criteria for initiation and engagement in SUD treatment, respectively. Little variation in initiation (2.8%) and engagement (3.9%) was collectively explained at the provider and clinic levels. No providers met the threshold for 70% reliability on an annual basis. Across varying assumptions of provider-level annual episode volume, clinics were unlikely to meet reliability thresholds. Factors consistently associated with IET included insurance type and history of behavioral health conditions. CONCLUSION: A low proportion of variation explained implies a larger number of episodes is needed to ensure reliable estimates. Stakeholders should apply IET measures cautiously, ensuring sufficient sample sizes to support robust measurement. Factors associated with a lower probability of meeting quality criteria can inform specific patients who may benefit from efforts to increase IET.