Developing and validating predictive models of electronic health record tool adoption in the pre-implementation stage

Abstract Background Many factors hinder adoption of implementation strategies and / or target evidence-based interventions. A gap in implementation science is the paucity of methodology for precisely assessing which medical practices will benefit most from a particular implementation strategy, how much support a medical practice might need, and if some medical practices could adopt evidence-based interventions on their own. Using machine learning, we developed and validated predictive models of the adoption of electronic health record (EHR)-related implementation strategies. Methods This retrospective study extracted data from an EHR-based cervical cancer screening (CVC) clinical decision tool for 310 community health centers from 5/1/2017 to 6/30/2019. Machine learning models (i.e., logistic lasso regression) were used to predict adoption (any tool use in the 12 months after tool rollout vs. none) and sustained use (≥ 1 tool use in the last quarter of the 12-month study period vs. < 1 tool use in this time period). Model performance was compared using the area under the receiver operating characteristics curve (AUC) from an external, withheld clinic sample. Results Prediction models showed good performance for adoption (AUC = 0.891, 95% CI = 0.801–0.981) and sustainability (AUC = 0.954, 95% CI = 0.889-1.000) of the CVC tool. Of the 23 EHR-derived clinic-level indicators used in the model, 18 (e.g., urbanicity) predicted adoption, and 19 (e.g., patient age) predicted sustainability of the CVC tool. Conclusions This work provides an important step toward efficiently tailoring and deploying implementation strategies for information technology innovations.